7 ms·

I'm probably nitpicking, but if you're using cat to pipe a single file into the sdtin of another program, you most likely don't need the cat in the first place,

I'm probably nitpicking, but if you're using cat to pipe a single file into the sdtin of another program, you most likely don't need the cat in the first place, you can just redirect the file to the process' stdin. Unless, of course, you're actually concatenating multiple files or maybe a file and stdin together. Disclaimer: I do cat-piping myself quite a bit out of habit, so I'm not trying to look down at the author or anything like that! :)

tyingq 8y ago
The "useless use of cat" was a repeated gripe on Usenet back in the day: http://porkmail.org/era/unix/award.html http://porkmail.org/era/unix/award.html
empath75 8y ago
People use cat to look at the file first, then hit up arrow, add a pipe, etc.
- longwave 8y ago
  Yes, this. Quite often I start writing out complex pipelines using head/tail to test with a small dataset and then switch it out for cat when I am done to run it on the full thing. And it's often not worth refactoring these things later unless you are really trying to squeeze performance out of them.
Leace 8y ago
I actually think that cat makes it more obvious what's happening in some cases. I had recently built a set of tools used primarily via pipes: (tool-a | tool-b | tool-c) and it looks clearer when I mock (for testing) one command (cat results | tool-b | tool-c) instead of re-flowing it just to avoid cat and use direct files.
toxik 8y ago
I think it's also a grammatical wart of shell syntax. Things going into a command are usually on the left, but piping in a file goes on the right.
- richardwhiuk 8y ago
  <file command | command | command is perfectly fine.
  - nine_k 8y ago
    The arrow now points backwards.
  - chrisfinazzo 8y ago
    Of course, if any of your commands prompt for input, you'll be disappointed that's not always as easy as it appears on the surface. Does anyone have a better way to do this kind of thing?
    - milkey_mouse 8y ago
      The standard is expect [1]. There are also libraries for many programming languages which perform a similar task, such as pexpect [2]. [1] https://core.tcl.tk/expect/index https://core.tcl.tk/expect/index [2] https://pexpect.readthedocs.io/en/stable/ https://pexpect.readthedocs.io/en/stable/
      - richardwhiuk 8y ago
        The better solution is to change the command so it expects programatic arguments / pass command line parameters. i.e. prefer `apt-get install -y` over `yes | apt-get install foo`
arendtio 8y ago
In fact, I don't like people optimizing shell scripts for performance. I mean, shell scripts are slow by design and if you need something fast, you choose the wrong technology in the first place. Instead, shell script should be optimized for readability and portability and I think it is much easier to understand something like 'read | change >write' than 'change <read >write'. So I like to write pipelines like this: cat foo.txt \ | grep '^x' \ | sed 's/a/b/g' \ | awk '{print $2}' \ | wc -l >bar.txt It might be not the most efficient processing method, but I think it is quite readable. For those who disagree with me: You might find the pure-bash-bible [1] valuable. While I admire their passion for shell scripts, I think they are optimizing to the wrong end. I would be more a fan of something along the lines of 'readable-POSIX-shell-bible' ;-) [1]: https://github.com/dylanaraps/pure-bash-bible https://github.com/dylanaraps/pure-bash-bible
- deleted 8y ago
  [deleted]
- fooblat 8y ago
  That syntax is very unusual from anything I've seen. I am also a fan of splitting pipelines with line breaks for readability, however I put the pipe on the end of each line and omit the backslash. In Bash, a line that ends with a pipe always continues on the next line. In any case, it's probably just a matter of personal taste.
  - zorga 8y ago
    I do it exactly as he does, it's the best looking to me; that's the cleanest way to break a line imho.
- Hello71 8y ago
  This is a very silly way of writing it though. grep|sed can almost always be replaced with a simple awk: awk '/^x/ { sub("a", "b"); print $2; }' foo.txt. This way, the whole command fits on one line. If it doesn't, put your awk script in a separate file and simply call it with "awk -f myawkscript foo.txt".
chrisfinazzo 8y ago
I would be remiss if I did not point out that calling said program cat is a misnomer. Instead of 'string together in a series' (the actual dictionary definition, which coincidentally, pipes actually do) it quickly became 'print whatever I type to the screen.' Of course, the example @arendtio uses is correct, because they obviously care about such things.
- Sharlin 8y ago
  Having separate commands for outputting the content of a single file and several files would, however, be an orthogonality violation. YMMV whether having a more descriptive name for the most common use of cat would be worth the drawback.
  - chrisfinazzo 8y ago
    It would fit in the broader methodology of 'single purpose tools that do their job well' or 'small pieces, loosely joined', but yes, probably too annoying to bother with.
maximilianburke 8y ago
I can see how it's redundant. But I use cat-pipes because I once mistyped the redirection and nuked my carefully created input file :) (Similarly, the first thing I used to do on Windows was set my prompt to [$p] because many years ago I also accidentally nuked a part of Visual Studio when I copied and pasted a command line that was prefixed with "C:\...>". Whoops.)
invsblduck 8y ago
Not nitpicking. Useless Use of Cat is an old thing: http://catb.org/jargon/html/U/UUOC.html http://catb.org/jargon/html/U/UUOC.html
assafmo 8y ago
I usually replace cat with pv and gets a nice progress bar and ETA :-)
dan-robertson 8y ago
For interactive use, I would like to point out that even better than this use of cat is less. If you pipe less into something then it forgets it’s interactive behaviour and works like cat on a single file. So: $ less foo | bar Is similar too: $ bar < foo Except that less is typically more clever than that and might be more like: $ zcat foo | bar Depending on the file type of foo.

tyingq 8y ago

The "useless use of cat" was a repeated gripe on Usenet back in the day: http://porkmail.org/era/unix/award.html http://porkmail.org/era/unix/award.html

empath75 8y ago

People use cat to look at the file first, then hit up arrow, add a pipe, etc.

longwave 8y ago
Yes, this. Quite often I start writing out complex pipelines using head/tail to test with a small dataset and then switch it out for cat when I am done to run it on the full thing. And it's often not worth refactoring these things later unless you are really trying to squeeze performance out of them.

Leace 8y ago

I actually think that cat makes it more obvious what's happening in some cases. I had recently built a set of tools used primarily via pipes: (tool-a | tool-b | tool-c) and it looks clearer when I mock (for testing) one command (cat results | tool-b | tool-c) instead of re-flowing it just to avoid cat and use direct files.

toxik 8y ago

I think it's also a grammatical wart of shell syntax. Things going into a command are usually on the left, but piping in a file goes on the right.

richardwhiuk 8y ago
<file command | command | command is perfectly fine.
- nine_k 8y ago
  The arrow now points backwards.
- chrisfinazzo 8y ago
  Of course, if any of your commands prompt for input, you'll be disappointed that's not always as easy as it appears on the surface. Does anyone have a better way to do this kind of thing?
  - milkey_mouse 8y ago
    The standard is expect [1]. There are also libraries for many programming languages which perform a similar task, such as pexpect [2]. [1] https://core.tcl.tk/expect/index https://core.tcl.tk/expect/index [2] https://pexpect.readthedocs.io/en/stable/ https://pexpect.readthedocs.io/en/stable/
    - richardwhiuk 8y ago
      The better solution is to change the command so it expects programatic arguments / pass command line parameters. i.e. prefer `apt-get install -y` over `yes | apt-get install foo`

arendtio 8y ago

In fact, I don't like people optimizing shell scripts for performance. I mean, shell scripts are slow by design and if you need something fast, you choose the wrong technology in the first place. Instead, shell script should be optimized for readability and portability and I think it is much easier to understand something like 'read | change >write' than 'change <read >write'. So I like to write pipelines like this: cat foo.txt \ | grep '^x' \ | sed 's/a/b/g' \ | awk '{print $2}' \ | wc -l >bar.txt It might be not the most efficient processing method, but I think it is quite readable. For those who disagree with me: You might find the pure-bash-bible [1] valuable. While I admire their passion for shell scripts, I think they are optimizing to the wrong end. I would be more a fan of something along the lines of 'readable-POSIX-shell-bible' ;-) [1]: https://github.com/dylanaraps/pure-bash-bible https://github.com/dylanaraps/pure-bash-bible

deleted 8y ago
[deleted]
fooblat 8y ago
That syntax is very unusual from anything I've seen. I am also a fan of splitting pipelines with line breaks for readability, however I put the pipe on the end of each line and omit the backslash. In Bash, a line that ends with a pipe always continues on the next line. In any case, it's probably just a matter of personal taste.
- zorga 8y ago
  I do it exactly as he does, it's the best looking to me; that's the cleanest way to break a line imho.
Hello71 8y ago
This is a very silly way of writing it though. grep|sed can almost always be replaced with a simple awk: awk '/^x/ { sub("a", "b"); print $2; }' foo.txt. This way, the whole command fits on one line. If it doesn't, put your awk script in a separate file and simply call it with "awk -f myawkscript foo.txt".

chrisfinazzo 8y ago

I would be remiss if I did not point out that calling said program cat is a misnomer. Instead of 'string together in a series' (the actual dictionary definition, which coincidentally, pipes actually do) it quickly became 'print whatever I type to the screen.' Of course, the example @arendtio uses is correct, because they obviously care about such things.

Sharlin 8y ago
Having separate commands for outputting the content of a single file and several files would, however, be an orthogonality violation. YMMV whether having a more descriptive name for the most common use of cat would be worth the drawback.
- chrisfinazzo 8y ago
  It would fit in the broader methodology of 'single purpose tools that do their job well' or 'small pieces, loosely joined', but yes, probably too annoying to bother with.

maximilianburke 8y ago

I can see how it's redundant. But I use cat-pipes because I once mistyped the redirection and nuked my carefully created input file :) (Similarly, the first thing I used to do on Windows was set my prompt to [$p] because many years ago I also accidentally nuked a part of Visual Studio when I copied and pasted a command line that was prefixed with "C:\...>". Whoops.)

invsblduck 8y ago

Not nitpicking. Useless Use of Cat is an old thing: http://catb.org/jargon/html/U/UUOC.html http://catb.org/jargon/html/U/UUOC.html

assafmo 8y ago

I usually replace cat with pv and gets a nice progress bar and ETA :-)

dan-robertson 8y ago

For interactive use, I would like to point out that even better than this use of cat is less. If you pipe less into something then it forgets it’s interactive behaviour and works like cat on a single file. So: $ less foo | bar Is similar too: $ bar < foo Except that less is typically more clever than that and might be more like: $ zcat foo | bar Depending on the file type of foo.