6 ms·
As well as anyone's general thoughts/experiences, I'd appreciate opinions on the error handling mechanism jb uses to detect errors in upstream jb processes that
by h4l 2y ago
As well as anyone's general thoughts/experiences, I'd appreciate opinions on the error handling mechanism jb uses to detect errors in upstream jb processes that jb is reading from.
Normally, detecting errors on the other end of a pipe requires care in a shell environment (e.g. retrospectively checking PIPESTATUS). I used an approach I've called Stream Poisoning. It takes advantage of the fact that control characters are never present in valid JSON. When jb fails to encode JSON, it emits a Cancel control character[1] on stdout. When jb encounters such a character in an input, it can tell the input it's reading from is truncated/erroneous. This avoids the typical problem of a pipe silently being read as an empty file.
I've got a page explaining this with some examples here: https://github.com/h4l/json.bash/blob/main/docs/stream-poisoning.md https://github.com/h4l/json.bash/blob/main/docs/stream-poiso... I can imagine using control characters in a text stream being rather controversial, but I feel it works quite well in practice.
[1]: https://en.wikipedia.org/wiki/Cancel_character https://en.wikipedia.org/wiki/Cancel_character
- throwawaynorway 2y agoWhat happens if the next program in the pipe is not jb? Does jb also exit with a code? For example `jb | jq`, where jq or a similar program discards the cancel character. (Away from pc, unable to check right now.)
- h4l 2y agoGood question! Yep, jb exits with non-zero: $ jb size:number=oops; echo $? json.encode_number(): not all inputs are numbers: 'oops' json(): Could not encode the value of argument 'size:number=oops' as a 'number' value. Read from inline value. ␘ 1 If you pipe the jb error into jq, jq fails to parse the JSON (because of the Cancel ctrl char) and also errors: $ jb size:number=oops | jq json.encode_number(): not all inputs are numbers: 'oops' json(): Could not encode the value of argument 'size:number=oops' as a 'number' value. Read from inline value. parse error: Invalid numeric literal at line 2, column 0 $ declare -p PIPESTATUS declare -a PIPESTATUS=([0]="1" [1]="4") So jq exits with status 4 here.