6 ms·

The problem with shell scripting is that nearly nobody is very, very good at it. The Steam bug doing an rm -rf / is an example, but it's very common for shell s

The problem with shell scripting is that nearly nobody is very, very good at it. The Steam bug doing an rm -rf / is an example, but it's very common for shell scripts to have horrible error handling and checks for important things. The shell is just not suitable for extremely robust programs. I would bet that 80%+ of people who think they're good at shell scripting... aren't.

freehunter 12y ago
Yeah but 80% of the people writing Java and think they're good aren't as well. And plenty of companies support Java. The answer isn't "don't use it", it's "train your programmers in the languages they use".
- kansface 12y ago
  The sorts of bugs people experience with Java mostly result in a crashed/stalled/hung process. Bash bugs erase your entire file system. The thing about Bash is that it is trivially easy to make these sorts of mistakes- the language just isn't suitable to general purpose scripting.
  - yourad_io 12y ago
    > Bash bugs erase your entire file system. EMPTY="" rm -rf $EMPTY/ Is this the kind of bug you're referring to? > The thing about Bash is that it is trivially easy to make these sorts of mistakes I fail to see how any other scripting language would have a different effect when you told it to do: system("rm -rf "+""+"/") > the language just isn't suitable to general purpose scripting. Yes, it is. Bash is deeper than it looks, but not by much. Learn how to handle errors and you'll be fine.
  - collyw 12y ago
    It shouldn't be able to erase your filesystem unless you are running as root or doing something equally stupid. That's pretty much common sense stuff for anyone that isn't a beginner.
    - gambiting 12y ago
      Yeah the "common sense stuff for anyone that isn't a beginner" argument is repeated ad nauseam, and even the largest companies make this mistake in their largest products. Take Valve - they should know how to write good code, right? And yet, last week an article was on top of HN, outlining how they put: "rm -rf '$STEAMROOT'/*" in their code, used to remove the library. But hey, no one checked if $STEAMROOT is not empty, so when it was for one user, Steam deleted all of his personal files, whole /home and /media directories next time it started. I'm not saying that command line tools shouldn't be used,but sometimes they are just too powerful for some users,and stupid mistakes like this happen.
      - anthony_d 12y ago
        You're right to an extent, but this isn't relevant to the Java vs Bash discussion. The largest companies make this kind of mistake in whatever language they happen to use. People delete data and screw things up in MapReduce jobs for Hadoop. A lot.
  - vidarh 12y ago
    If you're worried about that, don't give the script permissions to access your entire filesystem. Easily handled with separate users, cgroups, assorted containerisation, and more.
yourad_io 12y ago
> The problem with shell scripting is that nearly nobody is very, very good at it. The Steam bug doing an rm -rf / is an example The steam bug is an example of of utter incompetence; not of someone not being very, very good at it. Whoever is happy with shipping `rm -rf $VAR/` without extreme checking around it should get their computer driving license revoked. > The shell is just not suitable for extremely robust programs. Incorrect. "The shell" can go as robust as you can handle. In bash, `set -e` will kill your script if any of the sub-commands fail (although ideally you'll be testing $? (exit code of prev. op) at the critical junctions), `set -u` will error on usage of undefined variables, etc. A huge part of the "glue" that holds your favourite linux distro together is bash files. > I would bet that 80%+ of people who think they're good at shell scripting... aren't. The same probably goes for driving[1], this doesn't make cars any less robust. [1] http://en.wikipedia.org/wiki/Illusory_superiority http://en.wikipedia.org/wiki/Illusory_superiority
- yuubi 12y ago
  > rm -rf $VAR/ > / facepaw.jpg Without the trailing slash, null or undefined $VAR would cause an error instead of a request to delete all the things.
  - DougBTX 12y ago
    The Steam line was more like ` rm -rf $VAR/* `, eg, they didn't want to delete the $VAR directory. Still, ` rm -rf /* ` is no fun.
    - yourad_io 12y ago
      Part of the journey into Linuxdom is learning a healthy dose of fear for that command. I always pause over the enter key for a few seconds, even when I'm sure I haven't typo'ed.
- PhasmaFelis 12y ago
  > The same probably goes for driving[1], this doesn't make cars any less robust. I don't think I can imagine anything less robust than cars, in terms of the frequency and severity of operational failure. They're pretty much the deadliest thing we've ever invented that wasn't actually designed to kill people. It's actually a good example of the point developer1 was making: cars and shell scripts are perfectly safe if operated by highly competent people, and only become (extremely) dangerous when operated by incompetents, but in practice most operators are incompetent, in denial, and refuse to learn from others' mistakes.
sergiosgc 12y ago
> The shell is just not suitable for extremely robust programs. Absolute statements like this are usually wrong. This one does not escape the rule. When Linux distros init is mostly bash scripting, there is very little need to further prove that robust systems can be written in bash scripting without the language fighting the developer.
- vdaniuk 12y ago
  Wait, is it really a good argument for shell-based approach when all major distros are switching to the systemd due to the configuration/maintainability/boilerplate issues with bash init scripting?
  - sergiosgc 12y ago
    I'm not going into the systemd VS sysvinit discussion. For my argumentation, it is enough to recognize bash based sysvinit has been with us for circa 20 years with no stability problems.
jbergens 12y ago
I think most of was was written also applies to any normal programming language. You could write this in Python, Ruby, Javascript, Java or C# without any problems. The code would probably be easier to read also. The only special thing is the web page scraping that could be done by a library but the same thinking about scalability and the use of a single computer instead of a hadoop cluster still holds even if you're reading from file systems or databases.
MichaelGG 12y ago
Yeah I learned this trying to massage some data with Elasticsearch. curl -XDELETE host/index/type/$id ... Except $id didn't exist.

freehunter 12y ago

Yeah but 80% of the people writing Java and think they're good aren't as well. And plenty of companies support Java. The answer isn't "don't use it", it's "train your programmers in the languages they use".

kansface 12y ago
The sorts of bugs people experience with Java mostly result in a crashed/stalled/hung process. Bash bugs erase your entire file system. The thing about Bash is that it is trivially easy to make these sorts of mistakes- the language just isn't suitable to general purpose scripting.
- yourad_io 12y ago
  > Bash bugs erase your entire file system. EMPTY="" rm -rf $EMPTY/ Is this the kind of bug you're referring to? > The thing about Bash is that it is trivially easy to make these sorts of mistakes I fail to see how any other scripting language would have a different effect when you told it to do: system("rm -rf "+""+"/") > the language just isn't suitable to general purpose scripting. Yes, it is. Bash is deeper than it looks, but not by much. Learn how to handle errors and you'll be fine.
- collyw 12y ago
  It shouldn't be able to erase your filesystem unless you are running as root or doing something equally stupid. That's pretty much common sense stuff for anyone that isn't a beginner.
  - gambiting 12y ago
    Yeah the "common sense stuff for anyone that isn't a beginner" argument is repeated ad nauseam, and even the largest companies make this mistake in their largest products. Take Valve - they should know how to write good code, right? And yet, last week an article was on top of HN, outlining how they put: "rm -rf '$STEAMROOT'/*" in their code, used to remove the library. But hey, no one checked if $STEAMROOT is not empty, so when it was for one user, Steam deleted all of his personal files, whole /home and /media directories next time it started. I'm not saying that command line tools shouldn't be used,but sometimes they are just too powerful for some users,and stupid mistakes like this happen.
    - anthony_d 12y ago
      You're right to an extent, but this isn't relevant to the Java vs Bash discussion. The largest companies make this kind of mistake in whatever language they happen to use. People delete data and screw things up in MapReduce jobs for Hadoop. A lot.
- vidarh 12y ago
  If you're worried about that, don't give the script permissions to access your entire filesystem. Easily handled with separate users, cgroups, assorted containerisation, and more.

yourad_io 12y ago

> The problem with shell scripting is that nearly nobody is very, very good at it. The Steam bug doing an rm -rf / is an example The steam bug is an example of of utter incompetence; not of someone not being very, very good at it. Whoever is happy with shipping `rm -rf $VAR/` without extreme checking around it should get their computer driving license revoked. > The shell is just not suitable for extremely robust programs. Incorrect. "The shell" can go as robust as you can handle. In bash, `set -e` will kill your script if any of the sub-commands fail (although ideally you'll be testing $? (exit code of prev. op) at the critical junctions), `set -u` will error on usage of undefined variables, etc. A huge part of the "glue" that holds your favourite linux distro together is bash files. > I would bet that 80%+ of people who think they're good at shell scripting... aren't. The same probably goes for driving[1], this doesn't make cars any less robust. [1] http://en.wikipedia.org/wiki/Illusory_superiority http://en.wikipedia.org/wiki/Illusory_superiority

yuubi 12y ago
> rm -rf $VAR/ > / facepaw.jpg Without the trailing slash, null or undefined $VAR would cause an error instead of a request to delete all the things.
- DougBTX 12y ago
  The Steam line was more like ` rm -rf $VAR/* `, eg, they didn't want to delete the $VAR directory. Still, ` rm -rf /* ` is no fun.
  - yourad_io 12y ago
    Part of the journey into Linuxdom is learning a healthy dose of fear for that command. I always pause over the enter key for a few seconds, even when I'm sure I haven't typo'ed.
PhasmaFelis 12y ago
> The same probably goes for driving[1], this doesn't make cars any less robust. I don't think I can imagine anything less robust than cars, in terms of the frequency and severity of operational failure. They're pretty much the deadliest thing we've ever invented that wasn't actually designed to kill people. It's actually a good example of the point developer1 was making: cars and shell scripts are perfectly safe if operated by highly competent people, and only become (extremely) dangerous when operated by incompetents, but in practice most operators are incompetent, in denial, and refuse to learn from others' mistakes.

sergiosgc 12y ago

> The shell is just not suitable for extremely robust programs. Absolute statements like this are usually wrong. This one does not escape the rule. When Linux distros init is mostly bash scripting, there is very little need to further prove that robust systems can be written in bash scripting without the language fighting the developer.

vdaniuk 12y ago
Wait, is it really a good argument for shell-based approach when all major distros are switching to the systemd due to the configuration/maintainability/boilerplate issues with bash init scripting?
- sergiosgc 12y ago
  I'm not going into the systemd VS sysvinit discussion. For my argumentation, it is enough to recognize bash based sysvinit has been with us for circa 20 years with no stability problems.

jbergens 12y ago

I think most of was was written also applies to any normal programming language. You could write this in Python, Ruby, Javascript, Java or C# without any problems. The code would probably be easier to read also. The only special thing is the web page scraping that could be done by a library but the same thinking about scalability and the use of a single computer instead of a hadoop cluster still holds even if you're reading from file systems or databases.

MichaelGG 12y ago

Yeah I learned this trying to massage some data with Elasticsearch. curl -XDELETE host/index/type/$id ... Except $id didn't exist.