12 ms·
Linux Commands for Developers
- dazzawazza 14y agomight be better titled as "8 Unix commands every developer should know". They are all generic unix commands.
- protolif 14y agoTo continue reading, subscribe? seriously?
- talkingquickly 14y agotail -f filename should definitely make the list, essential for watching what's appended to log files in real time.
- alexfoo 14y agoI prefer:- tail -F filename as most of the logfiles I need to watch tend to wrap at some point and 'tail -f' doesn't check for inode changes. (-F is a non-standard option, it's there on GNU's [EDIT] tail binary and OS-X but not Solaris for example).
- talkingquickly 14y agoThanks, very useful - didn't know that!
- vacri 14y agoless +F filename does the same thing, but loses the following when the logfile gets rotated. Does the same thing happen to tail -f? (edit: alexfoo answered that tail -F tracks the file properly) one benefit to less +F is that you can cancel the following and read normally in less.
- masklinn 14y ago> one benefit to less +F is that you can cancel the following and read normally in less. Yeah, including doing backwards and forward searches, that is very handy
- tszming 14y agoA better title: 8 Linux Commands Every Developer Should Know [for log analysis]
- q_revert 14y agoI was expecting to see the first comment in here complaining about his use of 'cat', as in all of the examples his second argument could've easily taken a filename argument.. sort order.* is surely more elegant than cat order.* | sort which is fair enough, however, as it happens, i generally do end up using 'cat' in the way he's used it.. for such small jobs nobody can be genuinely worried about the overhead, and it comes down to a matter of taste.. personally, i find that using 'cat output | $command' helps to separate out the 'logic' of what i'm doing, if that makes sense.. also, again, purely as a matter of taste i'd prefer egrep 'Hardcover|Kindle' over grep "\(Kindle\|Hardcover\)" EDIT: (as alexfoo has pointed out, this isn't a proper AND as it worries about the order.. my bad, still useful though :D ) and as a sidenote, something i only found recently, but which is quite useful, a logical AND with egrep looks like egrep 'Hardcover.*Kindle'
- alexfoo 14y agoIndeed, I don't see why people get so upset (or pedantic) about what are, effectively, NOPs in command-lines. However, things change if you start adding certain options to sort:- sort -m order.* and cat order.* | sort -m are definitely not the same thing (for most input files at least).
- ralph 14y agoPerhaps because sending GiBs through read(2) and write(2) unnecessarily isn't a NOP?
- Athas 14y agoYou can do that sending while you're waiting for the disk to provide said GiBs. I also believe that useless uses of cat are often acceptable for readability (many novices are not familiar with redirecting standard input, particularly not as the first thing on a command line).
- ralph 14y agoWho said the GiBs need to be fetched from disk; they could already be in RAM. Even if not, it's still adding many system calls and context switches when the CPU could be doing other things; the machine isn't running just this one thing.
- reirob 14y agoI thought to learn something useful for programing (debugging, program runtime analysis, etc.). But instead the article is just about the generic commands cat, sort, grep, cut, sed, uniq, find and less. It is not really development related.
- antback 14y agoTotally agree! Wasted time. Everybody know these commands.
- Domenic_S 14y agoYou and I do, but surely you know a web dev or two that struggles to read log files meaningfully or doesn't know how to grep their html source for something? Send the article along, it's not always about you...
- jrajav 14y agoI disagree. They may not be obviously related to coding, though they'll probably end up being useful at some point anyway... But they're definitely useful for working with logs, working with datasets, working with config files, and a host of other development-related tasks. Just yesterday I was on a Windows machine and dearly felt the loss of sed and uniq. I have a task lined up for today to either find Windows alternatives or install msys.
- lloeki 14y agoUnless you want to show before/after context (-A/-B flags), one should grep before sort, not after. Less work to do.
- einhverfr 14y agoI came across something in fortune some time ago which you can find at: http://motd.ambians.com/quotes.php/name/linux_songs_poems/toc_id/1-1-31/s/8 http://motd.ambians.com/quotes.php/name/linux_songs_poems/to... I have found it to be surprisingly useful. Nobody uses all the commands but remembering that something like zcat exists can be extremely useful. Also remembering to pipe through sort before sending through uniq is helpful as well.
- gav 14y agoIn most cases you can replace "sort | uniq" with "sort -u".
- rimantas 14y agoOne guideline to keep in mind: > If the original title begins with a number or number + gratuitous > adjective, we'd appreciate it if you'd crop it. E.g. translate > "10 Ways To Do X" to "How To Do X," and "14 Amazing Ys" to "Ys." > Exception: when the number is meaningful, e.g. "The 5 Platonic Solids."
- yogione 14y agohow do I execute the out put of something like this: grep -i 'pattern' file | awk '{print $5}' | sed 's/^/cmd/g' I end up sending to a file, chmod, then run it at the shell.
- badboy 14y agoWrap it in $() to execute. And even if you pipe it into a file, no chmod is required. "sh file" works as it should
- slug 14y agoor use eval ~$ help eval eval: eval [arg ...] Execute arguments as a shell command. Combine ARGs into a single string, use the result as input to the shell, and execute the resulting commands. Exit Status: Returns exit status of command or success if command is null.
- michaelcampbell 14y agoIn bash... $(grep -i 'pattern' file | awk '{print $5}' | sed 's/^/cmd/g') ? Or surround in backticks `command which outputs text you want to run as a command` I prefer $() as they nest better. Or have I misunderstood your question?
- yogione 14y agoThat's it $() works. thanks.
- Domenic_S 14y agoxargs instead of that sed (I assume you're prepending your command to run there). grep -i 'pattern' file | awk '{print $5}' | xargs cmd
- pstadler 14y agoYou should really try ack (http://betterthangrep.com http://betterthangrep.com) as a replacement for grep.
- angusgr 14y agoOne I learned for the first time the other day is 'paste'. Good for people like me who never fully grokked awk, it joins lines from separate files into a single file. Say you have two files, one with lines of numbers: 1 2 3 ... and one with letters: A B C $ paste numbers letters 1 A 2 B 3 C Want CSV? $ paste -d, numbers letters 1,A 2,B 3,C Or, with '-s' you can join lines from inside the same file. For instance, you can sum numbers: $ paste -sd+ numbers 1+2+3 $ paste -sd+ numbers |bc 6 (Thanks to a Stack Overflow post somewhere for suggesting that one!) Useful example: the total resident memory size of all chromium processes: $ ps --no-headers -o rss -C chromium | paste -sd+ | bc 793180
- yesbabyyes 14y ago`paste` is also very useful together with stdin redirection and subshells. E.g.: paste <(ping 8.8.8.8) <(while true; do iwconfig wlan0 | grep "Bit Rate"; sleep 1; done)
- ralph 14y agoThey could drift. Better to do a one-ping ping and run iwconfig once in a loop.
- yesbabyyes 14y agoI was surprised and a little disappointed that `join(1)` didn't join the list! With the files in the example (order.out.log, order.in.log), to join every record on it's id, you would do something like: $ join -j 2 order.out.log order.in.log 111, 8:22:19 1, Patterns of Enterprise Architecture, Kindle edition, 39.99 8:22:20 Order Complete 112, 8:23:45 1, Joy of Clojure, Hardcover, 29.99 8:23:50 Order sent to fulfillment 113, 8:24:19 -1, Patterns of Enterprise Architecture, Kindle edition, 39.99 8:24:20 Refund sent to processing
- davidw 14y agoI think it's obligatory that someone writes 'strace' in threads about articles like this, so here goes. strace is fantastic for debugging certain categories of problem.
- cygwin98 14y agoAbsolutely, especially for those who do network programming under Unix/Linux. Combined with lsof, it can help a lot in troubleshooting issues such as deadlocks.
- pooriaazimi 14y agoI'm yet to find a decent introduction to strace (or dtrace). I'd appreciate if someone could point me to one...
- jswanson 14y agoHere's a post outlining how strace can be used to solve problems: https://blogs.oracle.com/ksplice/entry/strace_the_sysadmin_s_microscope https://blogs.oracle.com/ksplice/entry/strace_the_sysadmin_s... More from a sysadmin POV than a dev, but you'd probably still find it useful.
- gbog 14y agoIt says sed has basic stream editing capabilities. Basic. This guy should check, sed has very complex and powerful editing capabilities.
- zvrba 14y agocomm is another very useful command
- asdfprou 14y agoRegardless of the actual content, I applaud the "storytelling" way of presenting content. With a flow of examples that tie into each other the reader is given background context and can say to him/herself "I've had that problem before!", as I found myself doing.
- deleted 14y ago[deleted]
- dlsym 14y agosigh And I tought we were through with collections of trivial shell commands.
- djcb 14y agoHmmm, is this HN worthy? These commands are so basic that I don't expect any HN-reader using Unix systems not to know those. What about some slightly less basic ones that I'd think would be useful for many people. # less with syntax highlighting alias less="/usr/share/vim/vimcurrent/macros/less.sh" tailf # tail -f, but better & shorter mtr # check your connection (ie., traceroute with more info) htop # nice a colorful process listing (better than top) locate # search files by name -- that late-night disk thrashing is useful after all!
- stretchwithme 14y agoI found it useful. I was unaware of less.
- maw 14y agoI ask the programmers on my team, some of whom are quite junior and unexperienced, to send weekly status reports with an overview of what they did in the previous week, what they expect to do in the next, and other. Other is usually made up of areas of concern and interesting things that have made it onto one's radar. (You can put basically whatever you like in the other section. I'm still hoping somebody will send a joke.) Since I wouldn't ever ask them to do something I wouldn't do myself, I send these reports too. This link will go in my miscellany section this week. I think it'll be useful, and I wouldn't have ever found it or even considered including something like it had it not shown up on HN.
- thomaslangston 14y agoYes, it is HN worthy. Reviewing the basics is important and not all readers use Unix systems (frequently enough to remember the basics).
- masklinn 14y ago> tailf # tail -f, but better & shorter That's better about tailf, and why would I use it instead of the (far superior to tail) less +F?
- Evbn 14y agoIf you are going to use vim, why bit he pretending it is less? Just use vim.
- corford 14y agolsof -i always gets left out of these lists. It's really handy for quickly seeing what daemons are running (under which user) and on what interfaces they're bound to.
- dllthomas 14y agoThose are, indeed, 8 commands every developer should know. Calling them "Linux Commands" is a little weird - I don't think there's anything there not specified in POSIX, and I think they all appear on OS X and other unix systems.
- billsix 14y agoAgreed. Especially since he used 'find /Users -name "order*"', which means it was probably written on OS X.
- klochner 14y agoA better post + thread: "What are some time-saving tips that every Linux user should know?" http://www.quora.com/Linux/What-are-some-time-saving-tips-that-every-Linux-user-should-know http://www.quora.com/Linux/What-are-some-time-saving-tips-th... https://news.ycombinator.com/item?id=2361978 https://news.ycombinator.com/item?id=2361978