6 ms·

I also genuinely hate using jq. It is one of the only things that I rely heavily on AI.

by NSPG911 6mo ago

amelius 6mo ago
At that point why don't we ask the AI directly to filter through our data? The AI query language is much more powerful.
- latexr 6mo ago
  Because the output you get can have hallucinations, which don’t happen with a deterministic tool. Furthermore, by getting the `jq` command you get something which is reusable, fast, offline, local, doesn’t send your data to a third-party, doesn’t waste a bunch of tokens, … Using an LLM to filter the data is worse in every metric.
  - amelius 6mo ago
    You can use a local LLM and you can ask it to use tools so it is faster.
    - kelvinjps10 6mo ago
      There is hardware that is able to run jq but no a local AI model that's powerful enough to make the filtering reliable. Ex a raspberry pi
    - deleted 6mo ago
      [deleted]
    - sigseg1v 6mo ago
      "so it is faster" than what? A cloud hosted LLM? That's a pretty low bar. It's certainly not faster than jq.
  - alwillis 6mo ago
    I get that AI isn’t deterministic by definition, but IMHO it’s become the go-to response for a reason to not use AI, regardless of the use case. I’ve never seen AI “hallucinate” on basic data transformation tasks. If you tell it to convert JSON to YAML, that’s what you’re going to get. Most LLMs are probably using something like jq to do the conversion in the background anyway. AI experts say AI models don’t hallucinate, they confabulate.
    - tkclough 6mo ago
      Just because you haven't seen it hallucinate on these tasks doesn't mean it can't. When I'm deciding what tool to use, my question is "does this need AI?", not "could AI solve this?" There's plenty of cases where its hard to write a deterministic script to do something, but if there is a deterministic option, why would you choose something that might give you the wrong answer? It's also more expensive. The jq script or other script that an LLM generates is way easier to spot check than the output if you ask it to transform the data directly, and you can reuse it.
      - alwillis 6mo ago
        > but if there is a deterministic option, why would you choose something that might give you the wrong answer? Claude Code can use jq if it's installed on your system. Also, the data transformation is usually part of a larger workflow where an LLM is being used. And honestly, Claude is going to know jq better than 95% of developers who use it. jq can do a lot of things but it’s not the most intuitive tool to learn. An obvious best practice is to have the LLM use existing tools to confirm the correctness of its output.
    - djhn 6mo ago
      LLMs will often helpfully predict made up tokens for the content of the data fields. For 100% of jq use cases I have the data wouldn’t fit into context. But even for the smaller things, I have never, not even once, had an LLM not mangle data that is fed into it. Take a feed of blog posts (and select the first 50 or so just to give the model a fighting chance). I’ll give you 80% likelihood of the output being invalid JSON. And if you manage to get valid JSON out of it, the actual dates, times and text content will have changed.
- Shorel 6mo ago
  You really need to go and learn about the concept of determinism and why for some tasks we need and want deterministic solutions. It's an important idea in computer science. Go and learn.
  - amelius 6mo ago
    You need to learn to adapt to the real world where most things are not deterministic. Go and learn.
    - johnisgood 6mo ago
      That may be true, but do you not want determinism where possible, especially within this context, i.e. filtering data?
    - skipants 6mo ago
      Is your argument that the world isn't deterministic and so we should also apply nondeterminism to filtering json data?
    - Shorel 6mo ago
      I already know that. That's why we have deterministic algorithms, to simplify that complexity. You have much to learn, witty answers mean nothing here, particularly empty witty answers, which are no better than jokes. Maybe stand-up comedy is your call in life.
- imcritic 6mo ago
  Because the input might be sensitive. Because the input might be huge. Because there is a risk of getting hallucinations in the output. Isn't this obvious?
  - aduitsis 6mo ago
    ...and because it's going to burn a million times the energy of what jq would require.
- deleted 6mo ago
  [deleted]
vips7L 6mo ago
You should try nushell or PowerShell which have built ins to convert json to objects. It makes it so easy.
- bigstrat2003 6mo ago
  Second this. Working with nushell is a joy.
dannyobrien 6mo ago
I use the llm-jq plugin for Simon Willison's `llm` command line frontend for this: https://github.com/simonw/llm-jq https://github.com/simonw/llm-jq

I also genuinely hate using jq. It is one of the only things that I rely heavily on AI.

by NSPG911 6mo ago

I also genuinely hate using jq. It is one of the only things that I rely heavily on AI.

amelius 6mo ago

At that point why don't we ask the AI directly to filter through our data? The AI query language is much more powerful.

latexr 6mo ago
Because the output you get can have hallucinations, which don’t happen with a deterministic tool. Furthermore, by getting the `jq` command you get something which is reusable, fast, offline, local, doesn’t send your data to a third-party, doesn’t waste a bunch of tokens, … Using an LLM to filter the data is worse in every metric.
- amelius 6mo ago
  You can use a local LLM and you can ask it to use tools so it is faster.
  - kelvinjps10 6mo ago
    There is hardware that is able to run jq but no a local AI model that's powerful enough to make the filtering reliable. Ex a raspberry pi
  - deleted 6mo ago
    [deleted]
  - sigseg1v 6mo ago
    "so it is faster" than what? A cloud hosted LLM? That's a pretty low bar. It's certainly not faster than jq.
- alwillis 6mo ago
  I get that AI isn’t deterministic by definition, but IMHO it’s become the go-to response for a reason to not use AI, regardless of the use case. I’ve never seen AI “hallucinate” on basic data transformation tasks. If you tell it to convert JSON to YAML, that’s what you’re going to get. Most LLMs are probably using something like jq to do the conversion in the background anyway. AI experts say AI models don’t hallucinate, they confabulate.
  - tkclough 6mo ago
    Just because you haven't seen it hallucinate on these tasks doesn't mean it can't. When I'm deciding what tool to use, my question is "does this need AI?", not "could AI solve this?" There's plenty of cases where its hard to write a deterministic script to do something, but if there is a deterministic option, why would you choose something that might give you the wrong answer? It's also more expensive. The jq script or other script that an LLM generates is way easier to spot check than the output if you ask it to transform the data directly, and you can reuse it.
    - alwillis 6mo ago
      > but if there is a deterministic option, why would you choose something that might give you the wrong answer? Claude Code can use jq if it's installed on your system. Also, the data transformation is usually part of a larger workflow where an LLM is being used. And honestly, Claude is going to know jq better than 95% of developers who use it. jq can do a lot of things but it’s not the most intuitive tool to learn. An obvious best practice is to have the LLM use existing tools to confirm the correctness of its output.
  - djhn 6mo ago
    LLMs will often helpfully predict made up tokens for the content of the data fields. For 100% of jq use cases I have the data wouldn’t fit into context. But even for the smaller things, I have never, not even once, had an LLM not mangle data that is fed into it. Take a feed of blog posts (and select the first 50 or so just to give the model a fighting chance). I’ll give you 80% likelihood of the output being invalid JSON. And if you manage to get valid JSON out of it, the actual dates, times and text content will have changed.
Shorel 6mo ago
You really need to go and learn about the concept of determinism and why for some tasks we need and want deterministic solutions. It's an important idea in computer science. Go and learn.
- amelius 6mo ago
  You need to learn to adapt to the real world where most things are not deterministic. Go and learn.
  - johnisgood 6mo ago
    That may be true, but do you not want determinism where possible, especially within this context, i.e. filtering data?
  - skipants 6mo ago
    Is your argument that the world isn't deterministic and so we should also apply nondeterminism to filtering json data?
  - Shorel 6mo ago
    I already know that. That's why we have deterministic algorithms, to simplify that complexity. You have much to learn, witty answers mean nothing here, particularly empty witty answers, which are no better than jokes. Maybe stand-up comedy is your call in life.
imcritic 6mo ago
Because the input might be sensitive. Because the input might be huge. Because there is a risk of getting hallucinations in the output. Isn't this obvious?
- aduitsis 6mo ago
  ...and because it's going to burn a million times the energy of what jq would require.
deleted 6mo ago
[deleted]

vips7L 6mo ago

You should try nushell or PowerShell which have built ins to convert json to objects. It makes it so easy.

bigstrat2003 6mo ago
Second this. Working with nushell is a joy.

dannyobrien 6mo ago

I use the llm-jq plugin for Simon Willison's `llm` command line frontend for this: https://github.com/simonw/llm-jq https://github.com/simonw/llm-jq