12 ms·

> Shouldn't there be a sort in there before the uniq? Probably. uniq only removes duplicate lines when they're adjacent to each other. I doubt the git grep com

> Shouldn't there be a sort in there before the uniq? Probably. uniq only removes duplicate lines when they're adjacent to each other. I doubt the git grep command output has all the matches adjacent. I've read it's more efficient to use 'sort -u' instead of 'sort | uniq'. These days I only use the latter if I need uniq's -c to show the count of matches for each unique line.

tzs 5y ago
Careful: "sort -u" and "sort | uniq" are not equivalent. The former works on keys and the latter works on lines. It should be the same if you are sorting based on the whole line, but if you are sorting on part of the line it could make a difference. Consider this input 1 foo 2 foo 1 foo 2 bar 1 bar 2 bar For that "sort -u" and "sort | uniq" would give the same thing: 1 bar 1 foo 2 bar 2 foo But if you wanted to sort numerically, "sort -n -u" would not give the same result as "sort -n | uniq". The latter gives: 1 bar 1 foo 2 bar 2 foo but the former gives: 1 foo 2 foo The man page for GNU sort says of "-u": > with -c, check for strict ordering; without -c, output only the first of an equal run but "sort -n 1" gives: 1 bar 1 foo 1 foo 2 bar 2 bar 2 foo and so the equal runs are "1 bar", "1 foo", "1 foo" and similarly for the "2" lines, so I'd expect the output to be "1 bar" and "2 bar", not the "1 foo" and "2 foo" that it actually gives. The man page for BSD sort's explanation of "-u" explains what is going on: > Unique keys. Suppress all lines that have a key that is equal to an already processed one. This option, similarly to -s, implies a stable sort. If used with -c or -C, sort also checks that there are no lines with duplicate keys. Doing "sort -s -n 1" shows "1 foo" as the first "1" line and "2 foo" as the first "2" line, explaining why those are the two lines that make it past "-u".

tzs 5y ago

Careful: "sort -u" and "sort | uniq" are not equivalent. The former works on keys and the latter works on lines. It should be the same if you are sorting based on the whole line, but if you are sorting on part of the line it could make a difference. Consider this input 1 foo 2 foo 1 foo 2 bar 1 bar 2 bar For that "sort -u" and "sort | uniq" would give the same thing: 1 bar 1 foo 2 bar 2 foo But if you wanted to sort numerically, "sort -n -u" would not give the same result as "sort -n | uniq". The latter gives: 1 bar 1 foo 2 bar 2 foo but the former gives: 1 foo 2 foo The man page for GNU sort says of "-u": > with -c, check for strict ordering; without -c, output only the first of an equal run but "sort -n 1" gives: 1 bar 1 foo 1 foo 2 bar 2 bar 2 foo and so the equal runs are "1 bar", "1 foo", "1 foo" and similarly for the "2" lines, so I'd expect the output to be "1 bar" and "2 bar", not the "1 foo" and "2 foo" that it actually gives. The man page for BSD sort's explanation of "-u" explains what is going on: > Unique keys. Suppress all lines that have a key that is equal to an already processed one. This option, similarly to -s, implies a stable sort. If used with -c or -C, sort also checks that there are no lines with duplicate keys. Doing "sort -s -n 1" shows "1 foo" as the first "1" line and "2 foo" as the first "2" line, explaining why those are the two lines that make it past "-u".