6 ms·
"So that a truncated partial download doesn't end up executing half a script"
- oneepic 2y agoI read TFA. Why would a truncated partial download happen and still run the script?
- loeg 2y agoI believe 'curl XXX | sh' will start executing any complete shell statements in the input, before the full script is downloaded.
- LukeShu 2y agoThe most obvious answer is `curl | sh`. But also perhaps a network blip interrupting curl/wget, but the user failing to notice and going ahead and executing the file anyway.
- deleted 2y ago[deleted]
- wongarsu 2y agoA truncated download might happen for all sorts of reasons, like your internet connection dropping while you download the script. If you don't notice you might accidentally run an incomplete script and leave your system in some broken or at least confusing state. They wrapped everything in a main function to prevent that from happening
- loeg 2y agoBrowsers typically emit downloads to temporary files until they are complete, then rename them into the final location, to prevent this kind of issue.
- obi1kenobi 2y agoThe shell script starts with the following comment: # All the code is wrapped in a main function that gets called at the # bottom of the file, so that a truncated partial download doesn't end # up executing half a script.
- arp242 2y agoMost of these scripts have been doing that for years.
- obi1kenobi 2y agoI'm sure of it. And yet based on this rapidly getting to the front page, it seems like many of us are part of today's lucky ten thousand: https://xkcd.com/1053/ https://xkcd.com/1053/ Might be more than ten thousand, even, based on the reactions :)
- fuzzfactor 2y agoI had to do stuff like this over com ports using systems made before PC's.
- yau8edq12i 2y agoDon't pipe curl/wget a script to a shell without reading what you've downloaded. This should be common sense. Do `wget $url; most install.sh` and only if you're satisfied with what you read, execute `sh install.sh`.
- IncreasePosts 2y agoShe'll is complicated enough that I suspect it would be quite easy to trick anyone who does a cursory glance over the script, unless they are an absolute shell master.
- mirekrusin 2y agocurl -fsSL https://ollama.com/install.sh | sh && ollama run llama3 "Why it is bad to curl | sh?" ...for details.
- cassianoleal 2y agoAs opposed to downloading a binary install file?
- yau8edq12i 2y agoDistributors usually give you a way to verify that what you've downloaded is correct, usually through checksums, PGP signatures, code signing... You forego that if you pipe the script to your shell. What if you make a typo and somehow pipe an HTML document to your shell? If you're unlucky this could wreak havoc.
- cassianoleal 2y agoFrom the threat vectors you presented, I assume you already trust the vendor. That means you trust their installation script. You are, after all, going to run their binary after the installation! In this case, I assume the reason to inspect the script is not so much that the script might be doing something bad, but rather that you may have downloaded the wrong file to begin with. With that in mind: > Distributors usually give you a way to verify that what you've downloaded is correct The first thing is that not all software is downloaded from Linux distribution repositories. This technique doesn't work if you're just downloading an installer from a website or Github releases page, etc. Sure, many also provide you with a checksum that you need to manually verify, but the shell script in question can also do the same. In fact, it can help by automating the check after it inevitably downloads the application's binary package. In this case, the vector is you getting something different from what the vendor intended you to download. An example would be if your connection had been MITM'ed and a malicious package had been sent in its place. This is largely a non-issue these days with TLS certs everywhere, SNI, OCSP stapling and other protections that more or less ensure you're connected to the right server. > What if you make a typo and somehow pipe an HTML document to your shell? That's quite the bad luck! In this case, the user made a typo. Most `curl | bash` commands are copy-pasted from a website rather than typed out, so this is _mostly_ a non-issue as well. For those cases where the user typed the command and got it wrong, for it to become a problem, at least these 2 things need to be true: * the typoed URL actually downloaded something that the shell can interpret * there are commands in this downloaded document that actually wreak havoc to the system where they ran I fail to see a scenario where that would happen. Not that it's impossible, but it's so unrealistic that if it happened to me I might just shutdown the computer and go buy a lottery ticket!
- bombcar 2y agoI've seen scripts (self-extracting archives for Linux, for example) that checksum themselves either by some trickery, or just ignoring the first line after the shebang (which itself is the computed checksum of the rest of the file).
- Scaevolus 2y agoIncorporating an MD5 quine into a shellscript would be funny.
- seeknotfind 2y agoAnd a sha256 quine would be terrifying. :)
- cassianoleal 2y agoThat can be useful against download corruption but wouldn't do much against an actual attack (in this case, the attacker can just update the checksum).
- kazinator 2y agoThe problem is, the wrong party is doing the check (from a security point of view, not integrity). When we download a script from a remote domain we don't trust, we have to validate its checksum against the known one; we can't leave that to the script, which we don't trust.
- kevincox 2y ago99% of the time you are downloading from a domain that you do trust. This check is to detect corruption, not malice. But yes, if you were downloading from an untrusted mirror you would want to check the signature or trusted hash before running the script at all.
- joshspankit 2y agoIn this case we’re specifically talking about the possibility of a truncated script from a trusted source
- latchkey 2y agoIf only there was a way to transactionally run shell scripts such that if they don't complete fully, the changes are automatically reverted. Edit: cue the HN responses to use nix, and other solutions
- treve 2y agoPotentially sounds like a job for a BTRFS snapshot?
- rkeene2 2y agoNot really... although there have been attempts at doing this (e.g., Solaris) I've never seen it work out well for systems which could have multiple concurrent users. The main problem is that there are unrelated changes that could occur while performing an installation, for example adding a user while (but not part of) installing a package is a valid configuration right now but wouldn't be in the described scenario. Solaris handled this by having a config file that you could specify which files to copy between boot environments. Don't have /var/mail,/etc/passwd,/etc/shadow,/etc/hosts,.... in a different FS ? Better remember to copy it, or you lose your emails,users,hosts,etc The problem with this kind of failure mode is that it's silent and generally irreversible. There generally aren't tools to MERGE files that conflicted later on if you discover the issue while the old boot environment snapshot is still around. On the other hand, this is sort of how Docker (and Solaris zones) work -- and why you can't upgrade a container in a container-like way, you must replace the entire container (i.e., any upper-COW is lost, unless you export it and build something new atop it outside of any sort of reconciled process). On the other other hand, I've actually used BtrFS snapshots for exactly this successfully in exactly 1 case -- a read-only volume containing my OS files (12 files total; Kernel, Kernel Debug, Several Initramfs) for PXELINUX/EXTLINUX booting, for which can be atomically upgraded. Since it was read-only and the only operation supported on that volume was replacing the files, and it only had those 12 files, it was safe to do so.
- slimsag 2y agoMake curl | sh automatically upgrade the user's system to nix?
- deanCommie 2y agoA serious question for any Linux-heads here, no insult intended. How is it possible that there are ELEVEN different possible package managers that need to be supported by an installation script like this? I can understand that some divergences in philosophical or concrete requirements could lead to two, three, or four opinionated varieties, but ELEVEN? Does that mean that if I want to write an app that runs on Linux I should also be seeking to support 11 package managers? Or is there something unique about tailscale that would necessitate it? edit: Thank you for the responses so far, but noone has yet answered the core question: WHY are there eleven of them?
- Elucalidavah 2y agoIf you write an app that runs on Linux, you should support flatpak, and (for bonus points) nix. The rest should be done by the distros' maintainers.
- cassianoleal 2y agoFor something like Tailscale, running as a Flatpak would require the user to relax the sanbox boundaries, assuming it would work at all! It's probably easier to script up the installation via package manager. You also get the benefit of upgrades along with the rest of the system. Furthermore, I haven't seen a single instance of Flatpak being used to install applications on headless servers. I also don't know many sysadmins who would be happy that each application they install in their servers will come with a full set of dependencies rather than being dynamically linked to the base system.
- fsmv 2y agoNormally it's not your job to package it yourself the distro maintainers do that
- nick238 2y agoIf you're writing proprietary software and trying to get it out there so you can get paying users, it's your job.
- JonChesterfield 2y agoThis probably means you can edit the script while it's running without it falling over confusingly. Might cargo cult this pattern - I'm very prone to editing a build.sh while it runs.
- exe34 2y agoShot myself in the foot so many times with this!
- noisy_boy 2y agoA simple (not perfect) approach could be to have a comment containing a "unique" string on the last line and grep for it as the first check to ensure that the entire script has downloaded. #!/usr/bin/env bash set -u grep -wq '^# asfewdq42d3@asd$' $0 [ $? -ne 0 ] \ && echo "script is not complete - re-download" \ && exit 1 echo "script is complete" # asfewdq42d3@asd
- kevincox 2y agoThis doesn't work for something like `curl ... | sh`
- TrianguloY 2y agoAnother trick I've also seen is to enclose the whole script in a code block { code } That way, if the file is not fully loaded, the block will not end and the script will not parse
- Arcuru 2y ago...isn't that a common pattern? I'm pretty sure most install scripts I download and run already do that, though I don't run those very often.
- kazinator 2y agoI mean, could you put the main function at the top of the script, so that it calls later definitions? The problem is that the script could be truncated in such a way that it executes successfully. It defines a bunch of functions and then quits. If you're not checking for the success or failure of the download, you're probably not checking for the success or failure of the script; something is just going to assume the script worked.
- deleted 2y ago[deleted]
- kseifried 2y ago#!/bin/bash SHA512="485fe3502978ad95e99f865756fd64c729820d15884fc735039b76de1b5459d32f8fadd050b66daf80d929d1082ad8729620925fb434bb09455304a639c9bc87" # This line and everything later gets SHA512'ed and put in the above line. # To generate the sha512 simply: tail -n +3 [SCRIPTNAME].sh | sha512sum check_sha512() { # Compute the SHA512 hash of the script excluding the first two lines local current_sha=$(tail -n +3 "$0" | sha512sum | awk '{print $1}') # Compare the computed SHA512 hash with the predefined one if [[ "$current_sha" != "$SHA512" ]]; then echo "Error: SHA512 hash does not match!" exit 1 fi } # Call the function to perform the hash check check_sha512 # Rest of your script starts here echo "Script execution continues..." The idea is simple: if the first line get's mangled (#!/bin/bash) the script probably won't execute at all. If the second line gets mangled than obviously the SHA512 comparison won't work (variable name or value). Finally if the rest of the script gets mangled or truncated it won't SDHA512 the same and it'll cause the function to exit. For bonus points you can add a check if first line of script is exactly "#!/bin/bash" as well.
- o11c 2y agoIf the file is truncated after the function's closing brace, it will succeed but do nothing. If the file is truncated in the middle of the word `check_sha512` it will try to execute a hopefully-not-existing command. Wrapping in simple { braces } should fix this - if the brace is missing, you get a syntax error, and if present, you can execute the full thing, regardless of whether a trailing newline is available. This is admittedly bash-specific, so won't work for the linked script, but (subshell) doesn't cause too many problems Using a function and checking the SHA don't really add anything after these fixes. Checking the shebang is hostile to environments that install bash elsewhere. An almost-working possibility would be: exec some-interpreter -c 'commands' "$0" "$@" "" which will fail if the second ' is missing. The child interpreter can then check for later truncation by checking that at least 2 arguments were passed and the last one is an empty string. However, this is still prone to truncation before the -c.