9 ms·

Since no one seems to know about it, jq is described in great detail on the github wiki page [0]. That flattens the learning curve a lot. It's not as arcane as

Since no one seems to know about it, jq is described in great detail on the github wiki page [0]. That flattens the learning curve a lot. It's not as arcane as it seems. The touted claim that is fundamentally stateless is not true. jq is also stateful in the sense that it has variables. If you want, you can write regular procedural code this way. Some examples [1] The real problem of jq is that it is currently lacking a maintainer to assess a number of PRs that have accumulated since 2018. [0] https://github.com/stedolan/jq/wiki/jq-Language-Description https://github.com/stedolan/jq/wiki/jq-Language-Description [1] https://github.com/fadado/JBOL/blob/master/fadado.github.io/array/array.jq https://github.com/fadado/JBOL/blob/master/fadado.github.io/...

dilap 4y ago
From that page: > The jq documentation is written in a style that hides a lot of important detail because the hope is that the language feels intuitive. Yeah, not so much boys! Also, that disclaimer should really be at the top of the manual, with a link to the wiki, rather than vice-versa, as it is now. The wiki is like secret information -- "oh, hey, here's the page that actually tells you how it works!"
klysm 4y ago
I didn't realize jq was missing a maintainer, it's one of my most used CLI tools.
- skybrian 4y ago
  In this case it doesn't seem too critical? It means jq remains stable, which is probably what should happen once a tool like this gets a lot of users.
- ethanwillis 4y ago
  It really is a fundamental problem where lots of these important projects aren't maintained simply because the reality is the maintainers can't beat the economics of a lot of rich freeloaders having no real short term incentive to compensate these maintainers..
  - lenkite 4y ago
    Pity he quit before Github opened up sponsorships.
    - da39a3ee 4y ago
      How would Github sponsorship help pay for the time of someone at a finance company earning who knows where between 200-500k?
      - lenkite 4y ago
        My bad. I guess earning ~$100k is in-sufficient. https://changelog.com/posts/i-just-hit-100000-per-year-on-github-sponsors https://changelog.com/posts/i-just-hit-100000-per-year-on-gi...
        da39a3ee 4y ago
        Cool, I think I read that also! But, yeah it is on the low-end for a skilled engineer working for US companies, and also I'm thinking it's on the high-end / an outlier for github sponsors.
  - jahewson 4y ago
    It’s not though, because in this case the (ex-)maintainer works at a Wall St firm.
    - ethanwillis 4y ago
      This is exactly my point. Will he quit the paying job to work for free? How long could he maintain this for free with no other job, or with a job and additional free hours without running out of money or burning out?
      - lillecarl 4y ago
        But everyone can't realistically live a wealthy life off FLOSS tools either. The people who write these things are usually very talented and will make killer pay anywhere they work. Usually a cool thing is when companies sponsor them to work X% for them and Y% on the FLOSS tool.
jdnier 4y ago
Here's podcast interview with the creator of jq about what he's been working on at Jane Street: https://signalsandthreads.com/memory-management/ https://signalsandthreads.com/memory-management/
Beltalowda 4y ago
> It's not as arcane as it seems. The issue with jq is that I use it maybe once a month, or even less. The syntax is "arcane enough" that I keep forgetting how to use it because I use it so sporadically. In comparison awk – which I also don't use that often – has a much easier syntax that I can mostly remember. Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forgetting it.
- hiram112 4y ago
  Bingo. There are at least a dozen tools and languages and syntaxes that I've used sporadically over the years - awk, sed, bash, Mongo, perl, etc. I don't use them often enough to remember exactly how they work, and so I always have to spend a few hours reviewing manuals or old code repos or an O'Reilly book. But if I do end up using it for a few days in a row, it starts to make sense, and I improve each time I use it. But not with jq. It just does not make sense to my brain, no matter how many times I've had to use it. Every single time I need to use it, it requires finding some Stack Exchange or blog and just copying and pasting. Even after seeing the solution, rarely do I then really understand why or how it works. Nor can I often take that knowledge and apply it to similar problems. About the only other syntax or language that gives me such problems is Elastic Search DSL.
  - silon42 4y ago
    Same for me... everytime I have to lookup the basics... and I love awk,perl and xpath/xslt.
- ts0000 4y ago
  Interesting, for me it's the exact opposite. I've tried a couple of times to get into awk, but still find the syntax arcane.
  - Beltalowda 4y ago
    I don't know; I wouldn't presume to tell you what you do or don't find arcane, but once I understood the somewhat unusual flow of awk ("for every line, check if the line matches this condition, and if it does run this block of code") I found it's quite easy to work with. It's "arcane" in the sense that it has an implicit loop and that it's a specialized language for a very limited class of problems, but I found that for this limited class of problem it's surprisingly effective.
j1elo 4y ago
Sadly very few authors seem to acknowledge or even know that github wiki pages are not indexed by search engines so if it wasn't for third-party sites like github-wiki-see.page (which could stop working at any time) their contents would be undiscoverable by the very same people they are usually intended...
- oblio 4y ago
  What? That's crazy! Does Github block indexing?
  - beembeem 4y ago
    https://github.com/robots.txt https://github.com/robots.txt I don't see anything here about wiki specifically but maybe one of the rules hits wiki pages?
    - crazysim 4y ago
      They've moved from robots.txt to blocking by headers.
  - bckygldstn 4y ago
    There's more details on https://github-wiki-see.page/ https://github-wiki-see.page/ and https://github.com/github/feedback/discussions/4992#discussioncomment-1448177 https://github.com/github/feedback/discussions/4992#discussi... > we have also introduced an x-robots-tag: none in the http response header of Wiki pages > Abusive behavior in Wikis had a negative impact on our search engine ranking > GitHub is currently permitting a select criteria of GitHub Wikis to be indexed
adamgordonbell 4y ago
I found it hard to approach at first, but I think it was just the lack of material that worked through simple examples step by step. I ended up writing my own guide to it, that in my unbiased opinion makes it easier to get the point where in-depth examples and language descriptions are easier to understand. Edit: Oh, wow, it's even mentioned in this article. Maybe I should read before commenting. https://earthly.dev/blog/jq-select/ https://earthly.dev/blog/jq-select/
- sfink 4y ago
  I discovered jq after I wrote my own (extremely limited) version of it. I need it quite often, and yet I've never managed to get up the activation energy to learn enough for it to be useful. I need to have some notion of the computation model before anything is going to make sense to me. I hate learning things in completely disparate pieces that I need to memorize in hopes that someday it will just click together and I'll derive the underlying principles. Your guide was great for this. It stepped me through enough of the bare basics in a way that the underlying model was obvious. It didn't get me nearly far enough for many of the tasks that I need jq for, but it got me started and that's all I really needed. Everything additional that I need to learn becomes obvious in retrospect—"of course there's an operator for this, there kind of has to be!". Thank you!
deleted 4y ago
[deleted]

dilap 4y ago

From that page: > The jq documentation is written in a style that hides a lot of important detail because the hope is that the language feels intuitive. Yeah, not so much boys! Also, that disclaimer should really be at the top of the manual, with a link to the wiki, rather than vice-versa, as it is now. The wiki is like secret information -- "oh, hey, here's the page that actually tells you how it works!"

klysm 4y ago

I didn't realize jq was missing a maintainer, it's one of my most used CLI tools.

skybrian 4y ago
In this case it doesn't seem too critical? It means jq remains stable, which is probably what should happen once a tool like this gets a lot of users.
ethanwillis 4y ago
It really is a fundamental problem where lots of these important projects aren't maintained simply because the reality is the maintainers can't beat the economics of a lot of rich freeloaders having no real short term incentive to compensate these maintainers..
- lenkite 4y ago
  Pity he quit before Github opened up sponsorships.
  - da39a3ee 4y ago
    How would Github sponsorship help pay for the time of someone at a finance company earning who knows where between 200-500k?
    - lenkite 4y ago
      My bad. I guess earning ~$100k is in-sufficient. https://changelog.com/posts/i-just-hit-100000-per-year-on-github-sponsors https://changelog.com/posts/i-just-hit-100000-per-year-on-gi...
      - da39a3ee 4y ago
        Cool, I think I read that also! But, yeah it is on the low-end for a skilled engineer working for US companies, and also I'm thinking it's on the high-end / an outlier for github sponsors.
- jahewson 4y ago
  It’s not though, because in this case the (ex-)maintainer works at a Wall St firm.
  - ethanwillis 4y ago
    This is exactly my point. Will he quit the paying job to work for free? How long could he maintain this for free with no other job, or with a job and additional free hours without running out of money or burning out?
    - lillecarl 4y ago
      But everyone can't realistically live a wealthy life off FLOSS tools either. The people who write these things are usually very talented and will make killer pay anywhere they work. Usually a cool thing is when companies sponsor them to work X% for them and Y% on the FLOSS tool.

jdnier 4y ago

Here's podcast interview with the creator of jq about what he's been working on at Jane Street: https://signalsandthreads.com/memory-management/ https://signalsandthreads.com/memory-management/

Beltalowda 4y ago

> It's not as arcane as it seems. The issue with jq is that I use it maybe once a month, or even less. The syntax is "arcane enough" that I keep forgetting how to use it because I use it so sporadically. In comparison awk – which I also don't use that often – has a much easier syntax that I can mostly remember. Not entirely convinced by the zq syntax either though; it also seems "arcane enough" that I would keep forgetting it.

hiram112 4y ago
Bingo. There are at least a dozen tools and languages and syntaxes that I've used sporadically over the years - awk, sed, bash, Mongo, perl, etc. I don't use them often enough to remember exactly how they work, and so I always have to spend a few hours reviewing manuals or old code repos or an O'Reilly book. But if I do end up using it for a few days in a row, it starts to make sense, and I improve each time I use it. But not with jq. It just does not make sense to my brain, no matter how many times I've had to use it. Every single time I need to use it, it requires finding some Stack Exchange or blog and just copying and pasting. Even after seeing the solution, rarely do I then really understand why or how it works. Nor can I often take that knowledge and apply it to similar problems. About the only other syntax or language that gives me such problems is Elastic Search DSL.
- silon42 4y ago
  Same for me... everytime I have to lookup the basics... and I love awk,perl and xpath/xslt.
ts0000 4y ago
Interesting, for me it's the exact opposite. I've tried a couple of times to get into awk, but still find the syntax arcane.
- Beltalowda 4y ago
  I don't know; I wouldn't presume to tell you what you do or don't find arcane, but once I understood the somewhat unusual flow of awk ("for every line, check if the line matches this condition, and if it does run this block of code") I found it's quite easy to work with. It's "arcane" in the sense that it has an implicit loop and that it's a specialized language for a very limited class of problems, but I found that for this limited class of problem it's surprisingly effective.

j1elo 4y ago

Sadly very few authors seem to acknowledge or even know that github wiki pages are not indexed by search engines so if it wasn't for third-party sites like github-wiki-see.page (which could stop working at any time) their contents would be undiscoverable by the very same people they are usually intended...

oblio 4y ago
What? That's crazy! Does Github block indexing?
- beembeem 4y ago
  https://github.com/robots.txt https://github.com/robots.txt I don't see anything here about wiki specifically but maybe one of the rules hits wiki pages?
  - crazysim 4y ago
    They've moved from robots.txt to blocking by headers.
- bckygldstn 4y ago
  There's more details on https://github-wiki-see.page/ https://github-wiki-see.page/ and https://github.com/github/feedback/discussions/4992#discussioncomment-1448177 https://github.com/github/feedback/discussions/4992#discussi... > we have also introduced an x-robots-tag: none in the http response header of Wiki pages > Abusive behavior in Wikis had a negative impact on our search engine ranking > GitHub is currently permitting a select criteria of GitHub Wikis to be indexed

adamgordonbell 4y ago

I found it hard to approach at first, but I think it was just the lack of material that worked through simple examples step by step. I ended up writing my own guide to it, that in my unbiased opinion makes it easier to get the point where in-depth examples and language descriptions are easier to understand. Edit: Oh, wow, it's even mentioned in this article. Maybe I should read before commenting. https://earthly.dev/blog/jq-select/ https://earthly.dev/blog/jq-select/

sfink 4y ago
I discovered jq after I wrote my own (extremely limited) version of it. I need it quite often, and yet I've never managed to get up the activation energy to learn enough for it to be useful. I need to have some notion of the computation model before anything is going to make sense to me. I hate learning things in completely disparate pieces that I need to memorize in hopes that someday it will just click together and I'll derive the underlying principles. Your guide was great for this. It stepped me through enough of the bare basics in a way that the underlying model was obvious. It didn't get me nearly far enough for many of the tasks that I need jq for, but it got me started and that's all I really needed. Everything additional that I need to learn becomes obvious in retrospect—"of course there's an operator for this, there kind of has to be!". Thank you!

deleted 4y ago

[deleted]