6 ms·

How to destroy your OS with tar

98 points by vorakl 2y ago

deleted 2y ago
[deleted]
DerCommodore 2y ago
[flagged]
nightshift1 2y ago
Interesting read. Thanks for sharing. Maybe it is by lack of experience, but I always treat tarballs like a loaded gun. I extract then inside an empty subdirectory in my home first just to be sure and then move the data as required. It is no fun having to cleanup the mess left by an incorrect extraction.
- vorakl 2y ago
  This is a good and safe practice. And I think most people do it this way after cleaning up the mess of a badly built tarball at least once :)
  - nox101 2y ago
    I'd argue this is a design bug. Extracting into the current directory should either not be possible or be the exception (tar xvfz --current-directory) and not singling out tar here, unzip, pkunzip, etc all have this issue and have all cause people data loss and worse because if this default unsafe behavior
    - deleted 2y ago
      [deleted]
    - djbusby 2y ago
      Lots of things looks like a design bug today but were a prudent choice for the environment at the time. Like when I have an electrician at me house. Says the old way was dumb... but that was fully up to code in 1954. Time makes fools of us all.
      - nox101 2y ago
        That might be true but I've using arc, tar, pkzip since the 80s and even then I lost work and had to clean up on floppy disks because of this issue. I suppose the prudent thing is to list the files before decompressing
        djbusby 2y ago
        Sure (and me as well) but, design bug? Maybe just different expectations of the user. On that point, some heavy machinery took years before adding safety guards (some even required legislation before improving safety)
        nox101 2y ago
        Even `rm -rf *` warns you before just doing it.
    - 8n4vidtmkvmk 2y ago
      The fact that any program will overwrite files by default is also terrible.
    - eviks 2y ago
      Unless there is only one folder in the archive, and it's not overwriting anything, then it should be extracted into the current directory so you don't get nested dupes
- kennethrc 2y ago
  > I extract then inside an empty subdirectory in my home first ... AFTER a "tvf[jzx]", I hope
  - marcosdumay 2y ago
    What's the gain of -t if the extraction target is disposable? Does -x have some side effect that -t would list for you?
    - TheDong 2y ago
      It's nice to check if you're about to extract a 1MB tarball into 2TBs of data before actually running out of disk space. Most tar programs do prevent extracting tarballs containing absolute paths (like '/etc/passwd') and relative paths (like '../../../etc/passwd'), but older tar programs still allow that. And programs written in Go, because of course: https://github.com/golang/go/issues/55356 https://github.com/golang/go/issues/55356 Overall, if your HDD size is infinite and you're using GNU tar, or another recent tar, you can skip 't' I think before doing a '-C' extraction into some safe directory.
      - vorakl 2y ago
        How do you create a tar archive that "contains absolute paths"?
        10000truths 2y ago
        The tar file format doesn't prevent you from specifying absolute paths in the archive. It's up to the tool extracting the archive to reject/ignore such paths.
        vorakl 2y ago
        I asked about options for GNU tar because there is a bit of strange behavior. To add absolute paths to an archive, there is "-P" option, and man says it works only for creating archives: "Don't strip leading slashes from filenames when creating archives". To extract absolute paths from the archive, you need to add the "-C /" option, and although the tool says "tar: Strip leading `/' from member names", it will still extract it in the right place because the paths become relative and -C puts them in the root. However, if you add "-P" during the extraction (which is not mentioned in man), the "strip leading slashes" information disappears. So if this message bothers someone, "tar -C / -xPf file.tar" will cleanly extract absolute paths from the archive ;)
- chatmasta 2y ago
  Yeah, then I just hope I get the `mv` or `cp` right when I'm done, and don't end up with a directory full of files from the top part of the tar...
- zzzeek 2y ago
  I have 30 years of professional experience and with one off tarballs that I'm not deeply familiar with, this is usually what I do as well (certainly with a tarball that has a /usr like structure inside of it). You're good
- PhilipRoman 2y ago
  Most distros should have a useful Linux package "atool", containing the command "aunpack" which does the least surprising thing for all archive types (without creating duplicate root when it already exists).
  - BearOso 2y ago
    I believe the unar tool creates a containing directory by default.
- Gormo 2y ago
  Extracting archives directly into your system root as a superuser is in the same class of activity as piping curl output into your shell interpreter as a superuser: things that no one should ever do.
cbsks 2y ago
Any time you see a command starting with “sudo” and with a path of “/“ in the arguments, alarm bells should be going off. Here be dragons.
- vorakl 2y ago
  Yes, but someone has to do the dirty work ;)
  - millzlane 2y ago
    Appreciate the sacrifice
- bigstrat2003 2y ago
  Right, I would be more surprised if this didn't break your system. I never would've tried unpacking a tarball to my FS root in the first place.
- klyrs 2y ago
  What, you mean the increasingly popular install pattern curl ... | sudo bash might have pitfalls? Gasp.
  - mike256 2y ago
    You shouldn't do that. At least download the installer first, have a short look at it and decide if you like to run it. I personally avoid software whose manual suggests this type of install because I think of the developers as not that intelligent ones.
    - klyrs 2y ago
      Yeah, no, you shouldn't do that. Kinda my point in writing that, right? But it's terrifyingly commonplace and needs to stop.
      - prmoustache 2y ago
skynetv2 2y ago
Never untar anything to /. In this case, inexpensive, but a very valuable lesson.
PresidentWolf 2y ago
Been there. That was an interesting day.
silisili 2y ago
If I'm reading this right, using --no-overwrite-dir should have prevented this, I think. I'd still extract to a local folder as nonroot, though, so still the right conclusion.
- vorakl 2y ago
  --no-overwrite-dir (Preserve metadata of existing directories) This is a really good point! It solves the main problem. I'll add it to the article. Although that's what, unfortunately, is set by default (at least in GNU tar): --overwrite-dir (Overwrite metadata of existing directories when extracting (default).)
m463 2y ago
This reminds me once of using gparted on a disk. I got in and out of it a few times to make sure what I was going to do, then I launched it again (but I didn't add any arguments). Not only didn't it complain, it defaulted to my root disk, and I ended up destroying my partition table. Luckily the system was still running and I was able to backup everything before I shut the system down.
hcfman 2y ago
It’s funny how tar came out first. Then cpio came out which was a lot better but tar had the momentum and it never lost it. I still find cpio more controllable and use that as a preference. But cpio existed when I first used Unix in 1982/3
- eichin 2y ago
  and now there's pax (since POSIX looked at both of them and shuddered :-)
  - hcfman 2y ago
    My WOW with cpio is pretty elementary. I've never found anything in all my usage to shudder about. Possibly there are smarty pants usages in there that could make you shudder I don't know. But why choose smarty pants if elementary is perfectly satisfactory.
orliesaurus 2y ago
ah the joys of tar...the one time I got the input and output args confused and I lost a lot of data woops
prmoustache 2y ago
I never understood why so few people use `pax` instead of `tar`. It has always seemed more user friendly to begin with. With pax, the `-k` option will not overwrite existing files. Also not using `-p o` or `-p e` will not preserve rights. I don't know why you would extract and preserve rights to extract binaries in a chroot, usually you would want to control the permissions instead.
- rob74 2y ago
  The main reason is probably that tar is installed by default on 99% of Linux systems you might come across, while pax isn't...
  - prmoustache 2y ago
    Now that is funny, a few days ago I was downvoted to oblivion for stating the obvious: that nobody cares about POSIX anymore. And now you are telling me that all those downvoters are probably not following POSIX systems[1], either on their own desktop or their servers. [1] pax is the recommended archive utility in the POSIX shell&utilities section while tar and cpio aren't mentionned.
- silisili 2y ago
  Because tar is super versatile, old, and as the name implies, creates an archive. Useful for creating an archive to image something, and many times you want reproducible ownership and permissions. Tar can do all the same things, it just requires options either on the archiving or extraction end.
lm411 2y ago
Alternative title: How to destroy your OS by not understanding your OS. Don't take this the wrong way. Nice write up. :) ...... Sudo ... why would you sudo a command that you don't understand? RTFM ... all of the behaviour you experienced is well documented. There are no surprises. POSIX and UNIX utils are flexible and powerful. This is why we love them. Don't blame the hammer. Sandbox anything you extract and review it. Extract in a test environment. Extract without sudo and copy the files and permissions you need. tar -tv is simply tests the archive and outputs the paths. You ignored what it told you - that it would overwrite ./ :) #### Never ever do this! $ sudo tar -C / -xvfp xbps-static-latest.x86_64-musl.tar.xz Insanity. Cheers :)
- deleted 2y ago
  [deleted]
anthk 2y ago
Bubblewrap will allow you to create a chroot for xbps. No root needed. If $HOME/void_chroot is a void rootfs: https://docs.voidlinux.org/installation/guides/chroot.html#the-rootfs-method https://docs.voidlinux.org/installation/guides/chroot.html#t... bwrap --bind $HOME/void_chroot/ / --ro-bind /etc/resolv.conf /etc/resolv.conf --rw-bind /home/ /home/ --proc /proc --dev /dev /bin/sh Instead of xchroot, use bwrap.
doublerabbit 2y ago
I'm kind of annoyed that tar doesn't back up directories starting with a period unless you throw in a command flag. I wish I knew this before taking a backup and reformatting. Always test your backups I guess..
- vorakl 2y ago
  This is an interesting one. What was the version of tar? Was it gnu tar? I wonder, how did you create that archive? I'm asking this because I was trying to reproduce the same situation and everything seems to be working fine: $ tar -C /tmp/root3 -cvf test.tar . ./ ./.config/ ./.config/test ./.local/ ./var/ ./var/db/ ./var/db/xbps/ or even like this $ tar -cvf test2.tar root3/ root3/ root3/.config/ root3/.config/test root3/.local/ root3/var/ root3/var/db/ root3/var/db/xbps/ No special options were needed. But, if I do it this way, then, there are definitely missing all dot directories: $ tar -cvf test2.tar root3/* root3/usr/ root3/usr/bin/ root3/usr/bin/xbps-uunshare But this problem is not a tar's problem. That's only because "*" mask doesn't match dot files: $ echo root3/* root3/usr root3/var For that purpose, you need to clearly add a dot: $ echo root3/.* root3/.config root3/.local Thus, the solution might be $ tar -cvf test2.tar root3/.* root3/* root3/.config/ root3/.config/test root3/.local/ root3/usr/ root3/usr/bin/ But, I'd rather stick to "-C dir/" option instead of relying on "*" mask in this case.
  - doublerabbit 2y ago
    tar -cvf mytar.tar /home/myuser/* Was definitely the syntax I ran. I'm on FreeBSD so just tar.
    - vorakl 2y ago
      Then, this explains why. BTW, this is an expected Shell behavior, specified in the POSIX standard: If a filename begins with a <period> ( '.' ), the <period> shall be explicitly matched by using a <period> as the first character of the pattern or immediately following a <slash> character. The leading <period> shall not be matched by: * The <asterisk> or <question-mark> special characters https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13_03 https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... Interesting fact about the FreeBSD tar's origins: GNU tar was included as the standard system tar in FreeBSD beginning with FreeBSD 1.0. This is a complete re-implementation based on the libarchive(3) library. It was first released with FreeBSD 5.4 in May, 2005. https://man.freebsd.org/cgi/man.cgi?query=tar https://man.freebsd.org/cgi/man.cgi?query=tar My personal journey with FreeBSD began with version 5.3 (the first stable release on the 5th branch) in November 2004. I was completely unaware of such a significant tar change, and apparently I didn't care at the time. However, the entire 5th branch has been so revolutionary compared to the 4th branch that this change is just a drop in the bucket ;)

98 points by vorakl 2y ago

deleted 2y ago

[deleted]

DerCommodore 2y ago

[flagged]

nightshift1 2y ago

Interesting read. Thanks for sharing. Maybe it is by lack of experience, but I always treat tarballs like a loaded gun. I extract then inside an empty subdirectory in my home first just to be sure and then move the data as required. It is no fun having to cleanup the mess left by an incorrect extraction.

vorakl 2y ago
This is a good and safe practice. And I think most people do it this way after cleaning up the mess of a badly built tarball at least once :)
- nox101 2y ago
  I'd argue this is a design bug. Extracting into the current directory should either not be possible or be the exception (tar xvfz --current-directory) and not singling out tar here, unzip, pkunzip, etc all have this issue and have all cause people data loss and worse because if this default unsafe behavior
  - deleted 2y ago
    [deleted]
  - djbusby 2y ago
    Lots of things looks like a design bug today but were a prudent choice for the environment at the time. Like when I have an electrician at me house. Says the old way was dumb... but that was fully up to code in 1954. Time makes fools of us all.
    - nox101 2y ago
      That might be true but I've using arc, tar, pkzip since the 80s and even then I lost work and had to clean up on floppy disks because of this issue. I suppose the prudent thing is to list the files before decompressing
      - djbusby 2y ago
        Sure (and me as well) but, design bug? Maybe just different expectations of the user. On that point, some heavy machinery took years before adding safety guards (some even required legislation before improving safety)
        nox101 2y ago
        Even `rm -rf *` warns you before just doing it.
  - 8n4vidtmkvmk 2y ago
    The fact that any program will overwrite files by default is also terrible.
  - eviks 2y ago
    Unless there is only one folder in the archive, and it's not overwriting anything, then it should be extracted into the current directory so you don't get nested dupes
kennethrc 2y ago
> I extract then inside an empty subdirectory in my home first ... AFTER a "tvf[jzx]", I hope
- marcosdumay 2y ago
  What's the gain of -t if the extraction target is disposable? Does -x have some side effect that -t would list for you?
  - TheDong 2y ago
    It's nice to check if you're about to extract a 1MB tarball into 2TBs of data before actually running out of disk space. Most tar programs do prevent extracting tarballs containing absolute paths (like '/etc/passwd') and relative paths (like '../../../etc/passwd'), but older tar programs still allow that. And programs written in Go, because of course: https://github.com/golang/go/issues/55356 https://github.com/golang/go/issues/55356 Overall, if your HDD size is infinite and you're using GNU tar, or another recent tar, you can skip 't' I think before doing a '-C' extraction into some safe directory.
    - vorakl 2y ago
      How do you create a tar archive that "contains absolute paths"?
      - 10000truths 2y ago
        The tar file format doesn't prevent you from specifying absolute paths in the archive. It's up to the tool extracting the archive to reject/ignore such paths.
        vorakl 2y ago
        I asked about options for GNU tar because there is a bit of strange behavior. To add absolute paths to an archive, there is "-P" option, and man says it works only for creating archives: "Don't strip leading slashes from filenames when creating archives". To extract absolute paths from the archive, you need to add the "-C /" option, and although the tool says "tar: Strip leading `/' from member names", it will still extract it in the right place because the paths become relative and -C puts them in the root. However, if you add "-P" during the extraction (which is not mentioned in man), the "strip leading slashes" information disappears. So if this message bothers someone, "tar -C / -xPf file.tar" will cleanly extract absolute paths from the archive ;)
chatmasta 2y ago
Yeah, then I just hope I get the `mv` or `cp` right when I'm done, and don't end up with a directory full of files from the top part of the tar...
zzzeek 2y ago
I have 30 years of professional experience and with one off tarballs that I'm not deeply familiar with, this is usually what I do as well (certainly with a tarball that has a /usr like structure inside of it). You're good
PhilipRoman 2y ago
Most distros should have a useful Linux package "atool", containing the command "aunpack" which does the least surprising thing for all archive types (without creating duplicate root when it already exists).
- BearOso 2y ago
  I believe the unar tool creates a containing directory by default.
Gormo 2y ago
Extracting archives directly into your system root as a superuser is in the same class of activity as piping curl output into your shell interpreter as a superuser: things that no one should ever do.

cbsks 2y ago

Any time you see a command starting with “sudo” and with a path of “/“ in the arguments, alarm bells should be going off. Here be dragons.

vorakl 2y ago
Yes, but someone has to do the dirty work ;)
- millzlane 2y ago
  Appreciate the sacrifice
bigstrat2003 2y ago
Right, I would be more surprised if this didn't break your system. I never would've tried unpacking a tarball to my FS root in the first place.
klyrs 2y ago
What, you mean the increasingly popular install pattern curl ... | sudo bash might have pitfalls? Gasp.
- mike256 2y ago
  You shouldn't do that. At least download the installer first, have a short look at it and decide if you like to run it. I personally avoid software whose manual suggests this type of install because I think of the developers as not that intelligent ones.
  - klyrs 2y ago
    Yeah, no, you shouldn't do that. Kinda my point in writing that, right? But it's terrifyingly commonplace and needs to stop.
    - prmoustache 2y ago

skynetv2 2y ago

Never untar anything to /. In this case, inexpensive, but a very valuable lesson.

PresidentWolf 2y ago

Been there. That was an interesting day.

silisili 2y ago

If I'm reading this right, using --no-overwrite-dir should have prevented this, I think. I'd still extract to a local folder as nonroot, though, so still the right conclusion.

vorakl 2y ago
--no-overwrite-dir (Preserve metadata of existing directories) This is a really good point! It solves the main problem. I'll add it to the article. Although that's what, unfortunately, is set by default (at least in GNU tar): --overwrite-dir (Overwrite metadata of existing directories when extracting (default).)

m463 2y ago

This reminds me once of using gparted on a disk. I got in and out of it a few times to make sure what I was going to do, then I launched it again (but I didn't add any arguments). Not only didn't it complain, it defaulted to my root disk, and I ended up destroying my partition table. Luckily the system was still running and I was able to backup everything before I shut the system down.

hcfman 2y ago

It’s funny how tar came out first. Then cpio came out which was a lot better but tar had the momentum and it never lost it. I still find cpio more controllable and use that as a preference. But cpio existed when I first used Unix in 1982/3

eichin 2y ago
and now there's pax (since POSIX looked at both of them and shuddered :-)
- hcfman 2y ago
  My WOW with cpio is pretty elementary. I've never found anything in all my usage to shudder about. Possibly there are smarty pants usages in there that could make you shudder I don't know. But why choose smarty pants if elementary is perfectly satisfactory.

orliesaurus 2y ago

ah the joys of tar...the one time I got the input and output args confused and I lost a lot of data woops

prmoustache 2y ago

I never understood why so few people use `pax` instead of `tar`. It has always seemed more user friendly to begin with. With pax, the `-k` option will not overwrite existing files. Also not using `-p o` or `-p e` will not preserve rights. I don't know why you would extract and preserve rights to extract binaries in a chroot, usually you would want to control the permissions instead.

rob74 2y ago
The main reason is probably that tar is installed by default on 99% of Linux systems you might come across, while pax isn't...
- prmoustache 2y ago
  Now that is funny, a few days ago I was downvoted to oblivion for stating the obvious: that nobody cares about POSIX anymore. And now you are telling me that all those downvoters are probably not following POSIX systems[1], either on their own desktop or their servers. [1] pax is the recommended archive utility in the POSIX shell&utilities section while tar and cpio aren't mentionned.
silisili 2y ago
Because tar is super versatile, old, and as the name implies, creates an archive. Useful for creating an archive to image something, and many times you want reproducible ownership and permissions. Tar can do all the same things, it just requires options either on the archiving or extraction end.

lm411 2y ago

Alternative title: How to destroy your OS by not understanding your OS. Don't take this the wrong way. Nice write up. :) ...... Sudo ... why would you sudo a command that you don't understand? RTFM ... all of the behaviour you experienced is well documented. There are no surprises. POSIX and UNIX utils are flexible and powerful. This is why we love them. Don't blame the hammer. Sandbox anything you extract and review it. Extract in a test environment. Extract without sudo and copy the files and permissions you need. tar -tv is simply tests the archive and outputs the paths. You ignored what it told you - that it would overwrite ./ :) #### Never ever do this! $ sudo tar -C / -xvfp xbps-static-latest.x86_64-musl.tar.xz Insanity. Cheers :)

deleted 2y ago
[deleted]

anthk 2y ago

Bubblewrap will allow you to create a chroot for xbps. No root needed. If $HOME/void_chroot is a void rootfs: https://docs.voidlinux.org/installation/guides/chroot.html#the-rootfs-method https://docs.voidlinux.org/installation/guides/chroot.html#t... bwrap --bind $HOME/void_chroot/ / --ro-bind /etc/resolv.conf /etc/resolv.conf --rw-bind /home/ /home/ --proc /proc --dev /dev /bin/sh Instead of xchroot, use bwrap.

doublerabbit 2y ago

I'm kind of annoyed that tar doesn't back up directories starting with a period unless you throw in a command flag. I wish I knew this before taking a backup and reformatting. Always test your backups I guess..

vorakl 2y ago
This is an interesting one. What was the version of tar? Was it gnu tar? I wonder, how did you create that archive? I'm asking this because I was trying to reproduce the same situation and everything seems to be working fine: $ tar -C /tmp/root3 -cvf test.tar . ./ ./.config/ ./.config/test ./.local/ ./var/ ./var/db/ ./var/db/xbps/ or even like this $ tar -cvf test2.tar root3/ root3/ root3/.config/ root3/.config/test root3/.local/ root3/var/ root3/var/db/ root3/var/db/xbps/ No special options were needed. But, if I do it this way, then, there are definitely missing all dot directories: $ tar -cvf test2.tar root3/* root3/usr/ root3/usr/bin/ root3/usr/bin/xbps-uunshare But this problem is not a tar's problem. That's only because "*" mask doesn't match dot files: $ echo root3/* root3/usr root3/var For that purpose, you need to clearly add a dot: $ echo root3/.* root3/.config root3/.local Thus, the solution might be $ tar -cvf test2.tar root3/.* root3/* root3/.config/ root3/.config/test root3/.local/ root3/usr/ root3/usr/bin/ But, I'd rather stick to "-C dir/" option instead of relying on "*" mask in this case.
- doublerabbit 2y ago
  tar -cvf mytar.tar /home/myuser/* Was definitely the syntax I ran. I'm on FreeBSD so just tar.
  - vorakl 2y ago
    Then, this explains why. BTW, this is an expected Shell behavior, specified in the POSIX standard: If a filename begins with a <period> ( '.' ), the <period> shall be explicitly matched by using a <period> as the first character of the pattern or immediately following a <slash> character. The leading <period> shall not be matched by: * The <asterisk> or <question-mark> special characters https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13_03 https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V... Interesting fact about the FreeBSD tar's origins: GNU tar was included as the standard system tar in FreeBSD beginning with FreeBSD 1.0. This is a complete re-implementation based on the libarchive(3) library. It was first released with FreeBSD 5.4 in May, 2005. https://man.freebsd.org/cgi/man.cgi?query=tar https://man.freebsd.org/cgi/man.cgi?query=tar My personal journey with FreeBSD began with version 5.3 (the first stable release on the 5th branch) in November 2004. I was completely unaware of such a significant tar change, and apparently I didn't care at the time. However, the entire 5th branch has been so revolutionary compared to the 4th branch that this change is just a drop in the bucket ;)