6 ms·
Ask HN: Should I learn Awk or Perl? Or is that a bad question?
I find myself being totally lost on the command line when I need to do more complex stuff - a series of pipes between greps and cuts and the works. Recently, I finally took some time to try to learn some awk and it was life changing. I’ve been doing some awk puzzles, but it occurred to me that if I’ll be deliberately spending time learning a command line text processing tool, I may as well go the extra mile and learn perl, which I’ve heard is far more expressive than gawk.
I’m also noticing a lack of literature on Perl on the command line, I.e “perl -nae <some script>”.
Is awk just better for this use case?
- znpy 2y agoAwk for sure! The pdf for the original awk book is somewhere around the web, i learned awk from that and it’s mostly still good. I heard they made a new edition some time ago? Really worth reading! For perl… not sure. I learned a bit of perl in high school and never had a chance to use it, even at work where i occasionally see it being used from other people.
- gregjor 2y agoThe tools have a lot of overlap. No need to choose one over the other, you can learn both. I don't write Perl but knowing how to read it comes in handy. AWK usually does what I need.
- pvg 2y agoMaybe not so much a question of 'better' but perl is definitely bigger and multiheaded and does many, many more things (perl scientists were so preoccupied with whether or not they could, they didn't stop to think if they should). So you'll spend a lot more time (compared to awk) orienting yourself, beside the learning of the thing you want to know about. That could be a fun extra hobby or a waste of time, depending on your goals and inclinations.
- scrapheap 2y agoIf you're looking at learning Perl then you'll find the https://perldoc.perl.org/ https://perldoc.perl.org/ site very useful (you'll probably also have its content available on the command line via the perldoc command). If you want more information about running Perl on the command line then start with https://perldoc.perl.org/perlrun https://perldoc.perl.org/perlrun (or `perldoc perlrun` in your shell)
- fiedzia 2y agouse Python, so that other people can read it
- rurban 2y agoPerl for sure, because it's much better than awk. Also perl has a huge set of libraries, awk has nothing.
- pestatije 2y agoi wouldnt go for perl...id suggest the better question to be awk vs python, in which case id go for python
- Kim_Bruning 2y agoThis is a saner answer than you might think. In theory Python is the more verbose of these languages. But when I switched from Perl to Python way back when, my short scripts actually got shorter, were quicker and easier to write, and you could actually read them back and understand what they said! Of course both Perl and Python have changed a lot since I last compared them, so YMMV these days somewhat. I'd be interested to hear from a modern perl-er! (And: I do still use PCRE libraries sometimes :-P )
- srean 2y agoHow much time do you want to invest ? Perl is more powerful but learning and remembering all (or a sufficiently more powerful superset of awk's capabilities) of it is going to take time. The peculiarities of its syntax is certainly not a small set. All of awk on the other hand needs about 2 hours, if you have done some programming before.
- thesuperbigfrog 2y ago>> Perl is more powerful but learning and remembering all (or a sufficiently more powerful superset of awk's capabilities) of it is going to take time. The peculiarities of its syntax is certainly not a small set. I disagree. Perl's syntax is easy to learn for anyone who has used C-style syntax as found in C, C++, Java, Javascript, or C#: https://perldoc.perl.org/perlintro https://perldoc.perl.org/perlintro Perl's syntax is more similar to AWK and C-style languages than Python is. That said, there is nothing wrong with Python or AWK. They work great and have their places. But it is silly how many people treat Perl like it is impossible to learn or read without ever having used it or seeing where it is used.
- srean 2y agoIts OK to disagree. I am quite comfortable with C and C++ and still find it hard to hold all of Perl and its operators and naming rules in my head, to keep $s, @, % and .(dot) and quotes etc straight. Granted it makes types more obvious than say Python. AWK is smaller. There are fewer things to remember. Perhaps a good comparison would be the size of a Perl pocket guide and a AWK pocket guide. Perl and Python are much bigger subsets of AWK.
- thesuperbigfrog 2y ago>> I’m also noticing a lack of literature on Perl on the command line, I.e “perl -nae <some script>”. Searching for 'perl one liners' and the related term 'perl golf' gives many articles and books: https://www.perl.com/article/perl-one-liners-part-1/ https://www.perl.com/article/perl-one-liners-part-1/ https://www.perl.com/article/perl-one-liners-part-2/ https://www.perl.com/article/perl-one-liners-part-2/ https://catonmat.net/perl-one-liners-explained-part-one https://catonmat.net/perl-one-liners-explained-part-one https://catonmat.net/perl-one-liners-explained-part-two https://catonmat.net/perl-one-liners-explained-part-two https://catonmat.net/perl-one-liners-explained-part-three https://catonmat.net/perl-one-liners-explained-part-three https://catonmat.net/perl-one-liners-explained-part-four https://catonmat.net/perl-one-liners-explained-part-four https://catonmat.net/perl-one-liners-explained-part-five https://catonmat.net/perl-one-liners-explained-part-five https://catonmat.net/perl-one-liners-explained-part-six https://catonmat.net/perl-one-liners-explained-part-six https://catonmat.net/perl-one-liners-explained-part-seven https://catonmat.net/perl-one-liners-explained-part-seven http://novosial.org/perl/one-liner/ http://novosial.org/perl/one-liner/ https://www.oreilly.com/library/view/perl-one-liners/9781457185281/ https://www.oreilly.com/library/view/perl-one-liners/9781457...
- AnthOlei 2y agothis is amazing! I guess I was searing the wrong keywords. Thanks, I’ll evaluate these.
- Woshiwuja 2y agoi feel like awk is better for simple pipings and the likes, so i guess it depends on what you need to do. I find myself using awk a lot expecially with other programs
- _xk9i 2y agoThe case for Perl is just that it has no limits. You will never run out of tool if you go with Perl. That is what, I believe, makes the investment worthwhile. Why Perl gets such a bad rap, then? I'll tell you. I've used Perl for almost 30 years now, and I've never found any limits in the tool. This means every limit I found was _mine_. Many programmers feel bad when they find their own limits, but it doesn't have to be the case. You should be OK with finding your own limits, and be willing to overcome them, and strive to become a Perl power user, just like the founder, Larry Wall. If you choose this, to grow and improve and be better, and better, and better, then Perl is the best option. It's up to you.
- muzani 2y agoThis is good insight. Limits are often built into programming languages. Java is all about the limits. TS's selling point is literally the limits. As well as every opinionated language. I wonder if AI will make limitless languages easier to work with and maintain.
- shrimp_emoji 2y agoTo me, having to be the compiler and have 100% test coverage for typos is a limit. TS is a lame, misguided attempt to correct that to let people keep using such limited languages for things.
- _xk9i 2y agoI'm not against limits in programming languages. They help to reduce the complexity that have to be managed. Limited languages are suitable for those that are willing to manage only so much. That's why Java has no "goto". But Perl does offer "goto". Why? Because Larry Wall built Perl for himself, and he wanted the most powerful tool that he could conceive. That's exactly what Perl is. And I accept that, and I really want that. Every time I mess up with Perl, I don't blame my tool. I just laugh out loud and exclaim: "Thanks, Larry, for letting me know another way to not be up to my job". Perl is the magic wand. If you don't want to be a magician, you don't need it. But if you want to be a magician, you'll be thrilled to know Perl has three forms of "goto", not just one. This is what the textbook says about the third one: "The goto &NAME form is highly magical and sufficiently removed from the ordinary goto to exempt its users from the opprobium to which goto users are customarily subjected".
- gregw2 2y agoHaving learned both at an intermediate level, I use awk for structured data handling, Perl for textual pattern matching. Perl was inspired by awk, not the other way around. Perl is more general purpose.
- red-iron-pine 2y agothis was my understanding as well. perl grew out of sed/awk/bash, so might as well learn perl. that said, i found it gives a lot of 'foot-guns'.
- thesuperbigfrog 2y ago>> i found it gives a lot of 'foot-guns'. Most of those are for backward compatilibity with earlier versions. Recommend you place the following at the top of your Perl scripts: use strict; use warnings; These are recommended for all scripts: https://learn.perl.org/first_steps/ https://learn.perl.org/first_steps/
- asicsp 2y ago>noticing a lack of literature on Perl on the command line I have written a book on Perl one-liners with plenty of examples and exercises [0]. I've also written books on CLI text processing tools like grep, sed, awk and coreutils [1]. If you prefer just solving exercises, check out my interactive TUI apps [2]. >Is awk just better for this use case? It depends on the kind of tasks you'd need to solve. I generally prefer grep, sed and awk first. Perl helps if I need powerful regexp and other niceties like the huge number of built-in functions and access to third-party libraries. [0] https://learnbyexample.github.io/learn_perl_oneliners/ https://learnbyexample.github.io/learn_perl_oneliners/ [1] https://github.com/learnbyexample/scripting_course#ebooks https://github.com/learnbyexample/scripting_course#ebooks [2] https://github.com/learnbyexample/TUI-apps https://github.com/learnbyexample/TUI-apps
- AnthOlei 2y agoOh wow. I went though some of the perl book you linked and I was noticing the examples were really familiar; I then realized you were the same guy who wrote the awk book I have been going though. Your work is excellent! Thank you, I’ll buy a copy soon.
- thiht 2y agoI highly recommend awk because you can learn it in 20 minutes [1] So you can start by spending 20 minutes to learn awk, and then spend 20 years to learn Perl (and use awk in the mean time) [1]: https://ferd.ca/awk-in-20-minutes.html https://ferd.ca/awk-in-20-minutes.html
- thesuperbigfrog 2y agoPerl is far more capable than awk and more portable. The issue with awk is that there are multiple non-interoperable implementations: https://en.wikipedia.org/wiki/AWK#Versions_and_implementations https://en.wikipedia.org/wiki/AWK#Versions_and_implementatio... https://stackoverflow.com/questions/40409632/what-is-the-difference-between-the-different-awk-versions#43590842 https://stackoverflow.com/questions/40409632/what-is-the-dif... This makes awk scripts less portable than other text processing tools. awk is also not extensible--it can be awkward (heh) to adapt it to some problems and big scripts get difficult to wrangle. awk is great if your text processing problem is small. Sometimes small problems grow in which case Perl tends to be a better choice: https://stackoverflow.com/questions/366980/what-are-the-differences-between-perl-python-awk-and-sed#367014 https://stackoverflow.com/questions/366980/what-are-the-diff... People love to hate perl, but there is a reason why it was installed on all Unix and Linux systems by default and was so popular for web servers on the early Internet (e.g. Apache mod_perl https://perl.apache.org/ https://perl.apache.org/ and perl mason https://www.perl.com/pub/2002/12/11/mason.html/ https://www.perl.com/pub/2002/12/11/mason.html/ ). Perl also popularized regular expressions as a standard component of programming languages so much that "Perl-compatible regular expressions" are probably the most widely-used regex flavor versus POSIX or other regex variants: http://www.pcre.org/ http://www.pcre.org/ https://en.wikipedia.org/wiki/Regular_expression https://en.wikipedia.org/wiki/Regular_expression
- srean 2y ago> but there is a reason why it was installed on all Unix and Linux systems by default AWK has that one covered fair and square, even BusyBox has AWk.
- fargle 2y agoAWK is better suited to command line ad-hoc and shell script use. it is simple, straight-forward, and surprisingly powerful. most of the time sh+awk can replace sh+sed+cut+grep constructs. GNU awk (gawk) flavor is ubiquitous and it has a great manual [1]. and you have the AWK book [2] (not "free", but easily found). perl is more of a completely replacement for sh+awk for a standalone script/program. it was the first scripting language i know of that could do sockets networking and other complex things you'd previously need to write in c. it has extremely powerful text processing capabilities, like awk. but it's a big complicated, unusual scripting language. if you want a big complex full-featured scripting language to make a complete tool these days, why not use Python? if you only need really fancy text manipulation, maybe Perl, but i don't see that use a lot. [1] https://www.gnu.org/software/gawk/manual/html_node/index.html https://www.gnu.org/software/gawk/manual/html_node/index.htm... [2] https://awk.dev/ https://awk.dev/
- nextos 2y agoIMHO, Ruby is a much better Perl replacement than Python. Ruby is heavily inspired by Perl and lets you do a lot of work with simple one liners embedded in Bash scripts. It's also great for longer standalone programs and there is a larger community. Lots of interesting Perl-isms are also doable in Ruby.