6 ms·

why?

• faizshah 7y ago
  There was a post earlier on using command line tools instead of Hadoop for quick data processing. This shows a non-trivial example of how you could implement a complex data pipeline and an overview of some of the commands you could learn if you’re interested.
  • crmrc114 7y ago
    Yeah, this is a pretty cool post I never considered doing something like this in the shell. It seems silly to me that I forget how powerful basic tooling in the native shell can be.. sometimes I have a jackhammer and I forget that a sledgehammer will do the job just fine.
    • ahi 7y ago
      I've been there. 100s of lines into a ruby program then "oh yeah, cut sort grep"
      • e12e 7y ago
        Just for anyone else, you could probably end up in a similar corner with perl - but in both cases it's likely a case of "holding it wrong" - ruby borrows heavily from perl which borrowed heavily from shell with sed, awk, grep, cut and friends. So this kind of thing should be quite doable in a short ruby script - or a few short scripts - albeit written in "shell" style, with eg '-n or -p (wrap code in "while gets...end",-p with "puts _"), probably along with -a (automatically split lines). Its in some senses an entirely different dialect of ruby, though. Some examples here: https://github.com/learnbyexample/Command-line-text-processing/blob/master/ruby_one_liners.md https://github.com/learnbyexample/Command-line-text-processi...
• pmlnr 7y ago
  Because there's a big trend towards overengineering. I checked the Azure search solution, and my god it's complex. I ended up doing it in sqlite, and it perfect for a small fts document search engine. The article is similar: try to keep it simple; there are barely any cases where you'll use the real benefits of something lkke Azure search.
  • shakna 7y ago
    > I ended up doing it in sqlite, and it perfect for a small fts document search engine. It even has it as a feature! CREATE VIRTUAL TABLE something USING fts5(x, y, z)