10 ms·
JSON as a data-format should not have comments. JSON as a file-format should allow comments. The problem is this conflation between the two.
by model-15-DAV 2y ago
JSON as a data-format should not have comments. JSON as a file-format should allow comments. The problem is this conflation between the two.
- martin-adams 2y agoCan I confirm that the reason it's not preferred to have comments in data-formats is because it's to be machine read only and as such should be as efficient as possible and not contain information that wont be used? Seeing as I can only see the use case as a file format to be read/written by humans in the loop, then maybe the conversation should be about compiling the file format to a data format for compatibility outside of the user tooling.
- ur-whale 2y ago> is because it's to be machine read only Why did they bother making it text-only ASCII then ?
- DaiPlusPlus 2y ago> ASCII JSON is notable for making UTF-8 encoding a hard requirement. …which was pretty ballsy back in the mid-2000s. We were still fighting with Shift-JIS and Windows-1252. Excel didn’t add proper support for UTF-8 until depressingly recently.
- joemi 2y agoPretty sure they meant plaintext instead of ASCII.
- hinkley 2y agoLate 90’s I had to fix bugs in a shiftJIS implementation. And I couldn’t read a lick of Japanese. Still can’t. I don’t remember when I started pushing for utf-8 everywhere but it was “early” by most people’s standards, so I know what you mean. And one of the things that makes me dislike MySQL is that they have a field type called utf-8 that isn’t. And they didn’t fix it, they introduced a new type instead. So that footgun was still there for all to trigger. So mad.
- foldr 2y agoJSON does not require UTF-8 encoding.
- DaiPlusPlus 2y ago> JSON text exchanged between systems that are not part of a closed ecosystem MUST be encoded using UTF-8 https://datatracker.ietf.org/doc/html/rfc8259 https://datatracker.ietf.org/doc/html/rfc8259
- foldr 2y agoAh ok, fair enough. This is a more recent (2017) clarification of the standard which I hadn't seen. The original mid 2000s specification did not require UTF-8. > Previous specifications of JSON have not required the use of UTF-8 when transmitting JSON text. However, the vast majority of JSON-based software implementations have chosen to use the UTF-8 encoding, to the extent that it is the only encoding that achieves interoperability.
- DaiPlusPlus 2y agoThe original spec did require that all JSON decoders support UTF-8, though.
- foldr 2y agoHmm, not as I read it. It says that UTF-8 is the 'default' encoding. In context that just means that it's the encoding you assume if the first four octets don't match a pattern characteristic of one of the other encodings (when restricted to ASCII characters). See section 3 of https://datatracker.ietf.org/doc/html/rfc4627 https://datatracker.ietf.org/doc/html/rfc4627. The original RFC is vague, but I think the idea is that a fully conformant implementation would support all the encodings mentioned.
- hinkley 2y agoJSON wins because it can be casually inspected by people testing bizarre theories. The importance of this is lost on people who don’t treat triage as a skill that can be honed. I like to solve problems - or at least bringing them to me doesn’t result in a loss of status for either party. People notice this about me and bring me problems. Someone recently described to people what is essentially my process: the likelihood of the cause divided by the difficulty of verification. Partially sort and just start checking off assumptions. A lot of cheap but low probability options get shuffled higher, and just sending the wrong data is a common enough problem, especially with caching. And if it’s nearly free to look at the payload, it’ll get checked. If it isn’t people will try everything else to avoid it.
- deleted 2y ago[deleted]
- johannes1234321 2y agoThe argument is that comments are often used as an escape hatch from specified formats to carry further instructions. So you got a properly specified format and then want to do vendor&extensions but not break other implementations ... just make your extensions a comment. Then other parsers ignore it and you can do your thing. The idea is that this forces better formats. How well this works? Well, then I got an "x-comment" property or non-standard comments. Nonetheless. If people see the need to hack some extension in, they'll find a way.
- burnished 2y agoI think in the JSON case its because you can't have true comments, any comments are intrinsically part of the data structure, and you invite problems by including irrelevant information
- Kwpolska 2y agoThe problem is using JSON as a file format in the first place. It’s not designed for humans to edit. (Then again, it’s better than the Norway-sceptic YAML.)
- peterashford 2y agoI disagree. At least in an ought vs is sense: it's entirely the kind of format that I would create as an editable format. As witnessed by the fact that my workmates and I did create very nearly JSON previously as a file format in the 90s (but for C code programs)
- Kwpolska 2y agoFor a very narrow target audience (programmers), JSON is fine. If you want the file to be edited by anyone else, JSON is pain.
- deleted 2y ago[deleted]
- Cthulhu_ 2y agoBut it happens. 'npm install' will edit your json file, but so can I. That said, I don't like it as a config file read/written by humans.
- Jare 2y agoWhat example(s) of file format would you say are designed for humans to edit and still represent the kind of structured contents that json does?
- Kwpolska 2y agoI can’t think of anything that is not painful in some way.
- thayne 2y agoTOML, extensions of json like json5 and hjson, a bunch of lesser known formats for nested structures like NestedText, UCL, kdl, Eno,sdlang, eldf, etc. Also languages with some progrommatic capabilities like cue, dhall, jsonnet, nickel etc. Non of them are perfect, and some are less suitable for certain use cases than others. But IMO pretty much all of them are better for human editing than json, and in many cases yaml.
- pjc50 2y agoThe only reason it became popular is that conflation!
- lolinder 2y agoBut they're not two different formats—they're two different jobs being done by the same format. JSON as currently spec'd is honestly quite bad at both jobs, but the most rational defense of its use as a data format is that it's (mostly) human readable. Given that that's its main value proposition, what exactly is the reason for saying that JSON-as-data-format should not have comments? What do we lose if we allow them?
- sophacles 2y agoI think json should allow comments. But there are dangers there - look at how horribly comments get abused in code: * doctests are nonsense, just write tests. (doctests like rusts that just validate example snippets are the closest thing to good I've seen so far, but still make me nervous). * load bearing comments that code mangling/generation tools rely on (see a whole bunch of generated scripts in your linux systen - DO NOT EDIT BELOW THIS LINE) * things like modelines in editors that affect how programs interact with the code * things like html or xml comments that on parsing affect end user program logic. Comments can be abused, and in something like JSON on the wire I can see systems which take additional info from the comments as part of the primary data input. Often a completely different format... and you end up with something like the front-matter on your markdown files as found in static site generators. Point being, comments are not a purely benign addition.
- pulsarmx 2y agoHow is abusing comments any different than abusing a top-level property with a key like “__comment”?
- amatecha 2y agoat least a top-level metadata property can be explicitly defined in a .json.schema[0] and formalized, rather than being some kind of ad-hoc pre-processor step you have to evaluate before actually using the JSON data. I didn't even know about that approach before I read your comment but it instantly makes more sense to me in terms of maintainability and interoperability. [0] https://json-schema.org/ https://json-schema.org/
- jimmaswell 2y agoDisallow comments and now you just have {"comment": "the quick brown fox.."}, the worst of both worlds.
- hombre_fatal 2y agoThat's a harmless example and a tiny price to pay. What no-comments saved us from was stuff like this in our data interchange: { "count": 123 // bigint "price": 10.99 // @precision=2 "date": "2024-08-12" // @format=YY-MM-dd "data": /* !transform(rot13) */ "uryyb" "storage": 5 // Unit(TB) } And who knows what deeper layers of hell we avoided. Frankly, VSCode shows that all this time people were complaining about no comments in JSON config and how hard it was to write config in JSON, they could have just written their apps to strip comments at read time. So we do have the best of both worlds.
- bearjaws 2y agoYour example is perfect, I'm stealing this for the next time JSON comments comes up.
- codedokode 2y ago> JSON as a file-format should allow comments. JSON is awful for writing manually because it requires typing too many quotes, commas etc. I think JSON is meant to be machine-generated and machine-read and therefore doesn't need any comments.
- leptons 2y agoYou're a programmer and you're against writing quotes and commas? You must really hate coding. I've never found JSON to be too much typing.
- xnorswap 2y agoIf you're entirely machine writing and reading, but still want to be human-legible, then XML does a much better job while also allowing for schema.
- nivertech 2y agoThinking from the first principles: 1. comments are metadata (specifically Human/LLM-readable metadata vs machine-readable metadata) 2. general-purpose data formats should support metadata
- Someone 2y agoI think it’s more “it would be nice if JSON intended to be read or written by humans allowed comments”.