7 ms·
Bash’s built-in wait is also handy when you want quick and simple parallelism. http://tldp.org/LDP/abs/html/x9644.html http://tldp.org/LDP/abs/html/x9644.html
by termie 9y ago
Bash’s built-in wait is also handy when you want quick and simple parallelism. http://tldp.org/LDP/abs/html/x9644.html http://tldp.org/LDP/abs/html/x9644.html
- philsnow 9y agoI do this pattern a lot for x in $(seq 1 10); do long_process file${x} & done; wait It doesn't take care of not saturating my cpu or anything; when I need to care about that then I try to remember how to use parallel.
- enriquto 9y agoit's actually easier using parallel : for x in ...; do echo long_process file$x ; done | parallel -j 8 EDIT: and in your case, it is even easier with xargs, e.g.: ls files* | xargs -n 1 -P 8 long_process
- sigjuice 9y agoWouldn't this fail if your file names have white space?
- Anthony-G 9y agoParsing `ls` is never a good idea[1]. A more robust way to use globbing in a POSIX shell would be something like this: printf "%s\0" files* | xargs -0 -n 1 -P 8 long_process 1. http://mywiki.wooledge.org/ParsingLs http://mywiki.wooledge.org/ParsingLs
- enriquto 9y agoI disagree with the article that you linked. Filenames are variable names that you get to choose. Half of the game is won by chosing them wisely, so that they are convenient to use.
- enriquto 9y agoyes, and rightfully so. If you don't use silly filenames you can write simpler scripts.
- kwhitefoot 9y agoYou don't always have the luxury of choosing the names that the files have.
- enriquto 9y agoI actually do. In the rare cases where I have to work with odd filenames, it is easier to rename those files than to change my clean scripts.
- SamJson 9y agoAhh, I see a sysadmin with users! :)
- kwhitefoot 9y agoWell, ex-sysadmin for many years now. But yes, real world and all that.
- flatfilefan 9y agoI use this template to deal with reasonable filenames: find ... -print0|parallel --gnu -X -0 -n 1 your_command "{}" \; And sometimes the names are so screwed that they have to be renamed first, then this can give you a reasonable new name $(echo $(basename "{}")|tr '[[:blank:]]' '_'| tr -cd '\/.[[:alnum:]]_-' )
- SamJson 9y agoI would do: seq 10 | parallel long_process file{} Or simply: parallel long_process ::: file*