7 ms·
Tarsnap gives deduplication, compression, and encryption. Some technical background on how it does this here: http://www.tarsnap.com/efficiency.html http://www
by keyist 15y ago
Tarsnap gives deduplication, compression, and encryption. Some technical background on how it does this here: http://www.tarsnap.com/efficiency.html http://www.tarsnap.com/efficiency.html
There is a detailed breakdown provided as well. Sample output:
Total size Compressed size
All archives 697 MB 323 MB
(unique data) 215 MB 100 MB
This archive 148 MB 67 MB
New data 17 MB 5.2 MB
Edit: Don't want to misrepresent Tarsnap's granularity. The print-stats option gives size in bytes. Above output was generated with the --humanize-numbers option.
- nknight 15y agoI assume this is only deduplication within a single account? Because I don't see anything there on how one would provide dedupe across crypto domains (which, to the best of my knowledge, should be theoretically impossible if the crypto is done right).
- cperciva 15y agoCorrect.
- pbhjpbhj 15y agoJust been looking at Tarsnap. It looks good but I'm curious what happens if the author/creator/owner/maintainer (singular) gets run over by a bus?
- cperciva 15y agoI do my best to avoid busses. (I've always been more of a fan of switched point-to-point connections than busses, to tell the truth.) In all seriousness, if I get hit by a bus Tarsnap probably won't live on without me -- but it runs itself smoothly enough that there should be plenty of time for people to download their data.
- plinio_silva 15y agoCan I automatically sync files using it? I work in multiple places and if I have to manually run a sync operation before I leave, I'm certain to forget or be unable to at some occasion, and then I'm screwed.
- cperciva 15y agoTarsnap doesn't do sync. Doing sync while maintaining security is a much harder problem: Tarsnap's design requires each system to know what's on the server before uploading or deleting blocks, and that breaks if you have multiple systems accessing the same storage space.
- cperciva 15y agoMore sample output, from a heftier machine, in case anyone is curious: Total size Compressed size All archives 3321059166180 1113881916696 (unique data) 116996664943 27182261608 This archive 13831036740 4855451330 New data 336950399 62343192 (Yes, that's 3.3 TB deduplicated and compressed down to 27 GB.)