6 ms·
Sort will actually externally sort blocks into temp files and merge them. Adjusting this block size can help with thrashing. Awk may still be better for uniqui
by stevvooe 14y ago
Sort will actually externally sort blocks into temp files and merge them. Adjusting this block size can help with thrashing.
Awk may still be better for uniquification.
- enigmo 14y agoWhat about 'sort -u' ?
- stevvooe 14y agoI'm not sure. I haven't closely studied the difference between each algorithm. My guess would be that sort -u would perform better as the data set gets larger with a good block size setting because it does do an external sort. Cardinality would also affect the performance. If the unique set handily fits in memory, an external sort on a large data set wouldn't be very efficient.