7 ms·

Oj author here. While it's flattering to have Oj be the standard to beat I'd like to point out that most of the issues with Oj revolve around the JSON gem and R

Oj author here. While it's flattering to have Oj be the standard to beat I'd like to point out that most of the issues with Oj revolve around the JSON gem and Rails doing a monkey patch dance and Oj trying to keep pace with the changes. The Oj.mimic_JSON attempts to replace the JSON gem and only replaces the monkey patches made by that gem. The preferred approach for Oj outside of trying to mimic the JSON gem to to never monkey patch. That approach is used in all other modes that are not mimicking the JSON gem or Rails. I should point out that other Oj modes perform much better than the JSON gem and Rails modes.

byroot 2y ago
> I should point out that other Oj modes perform much better than the JSON gem Which modes are that? https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj-modes https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj... I tried: Oj.dump(obj, mode: :strict) and a few others and none seemed faster than `json 2.9.1` on the benchmarks I use. Edit: Also most of these mode simply aren't correct in my opinion: >> Oj.dump(999.9999999999999, { mode: :compat }) => "999.9999999999999" >> Oj.dump(999.9999999999999, { mode: :strict }) => "1000"
- peterohler 2y ago
  Using the benchmarks in the Oj test directory Oj has a slight advantage over the core json for dumping but not enough to make much difference. The comparison for Oj strict parsing compared to the core json is more substantial as 1.37 times faster. The benchmarks use a hash of mixed types included some nested elements. The callback parsers (Saj and Scp) also show a performance advantage as does the most recent Oj::Parser. As for the dumping of floats that are at the edge of precision (16 places), Oj does round to to 15 places if the last 4 of a 16 digit float is "0001" or "9999" if the float precision is not set to zero. That is intentional. If that is not the desired behavior and the Ruby conversion is preferred then setting the float precision to zero will not round. You picked the wrong options for your example. I would like to say that the core json has a come a very long way since Oj was created and is now outstanding. If the JSON gem had started out where it is now I doubt I would have bothered writing Oj.
  - byroot 2y ago
    > Using the benchmarks in the Oj test directory I'm sorry, but I've looked for a while now, and I can't seem to identify the benchmark you are mentioning. I suspect it's the one John took for his benchmark suite? [0] > Oj has a slight advantage over the core json for dumping but not enough to make much difference I'd be curious to see which benchmark you are using, because on the various ones included in ruby/json, Oj is slightly slower on about all of them: https://gist.github.com/byroot/b13d78e37b5c0ac88031dff763b3bbd2 https://gist.github.com/byroot/b13d78e37b5c0ac88031dff763b3b..., except for scanning strings with lots of multi-byte characters, but I have a branch I need to finish that should fix that. > The comparison for Oj strict parsing compared to the core json is more substantial as 1.37 times faster Here too I'd be curious to see your benchmark suite because that doesn't match mine: https://gist.github.com/byroot/dd4d4391d45307a47446addeb77748dc https://gist.github.com/byroot/dd4d4391d45307a47446addeb7774... > The callback parsers (Saj and Scp) also show a performance advantage as does the most recent Oj::Parser. Yeah, callback parsing isn't something I plan to support, at least not for now. As for Oj::Parser, `ruby/json` got quite close to it, but then @tenderlove pointed to me that the API I was trying to match wasn't thread safe, hence it wasn't a fair comparison, so now I still bench against it, but with a new instance every time: https://github.com/ruby/json/pull/703 https://github.com/ruby/json/pull/703. > You picked the wrong options for you example. No, I picked them deliberately. That's the sort of behavior users don't expect and can be bitten by. As a matter of fact, I discovered this behavior because one of the benchmark payloads (canada.json) doesn't roundtrip cleanly with Oj's default mode, that's why I benchmark against the `:compat` mode. IMO truncating data for speed isn't an acceptable default config. [0] https://github.com/jhawthorn/rapidjson-ruby/blob/518818e6768c516f18cd78b095e603bf8b323bd6/benchmark/parser.rb#L36 https://github.com/jhawthorn/rapidjson-ruby/blob/518818e6768...
    - peterohler 2y ago
      The strict mode benchmarks for Oj are in the test/perf_strict.rb. Others are are in perf_*.rb. If callback parsing is not supported that's fine. Oj does support callback parsing as it allows elements in a JSON to be ignored. That save memory, GC, and performance. Your choice of course just as including callback parsers is a choice for Oj. Ok, so you picked options that you knew would fail. Again you choice but there are certainly others that would trade a slight improvement in performance to not have 16+ significant digits. It's a choice. You are certainly entitled to you opinion but that doesn't mean everyone will share them. I'm not sure what platform you are testing on but i'm sure there will be variations depending on the OS and the hardware. I tested on MacOS M1.
- Twirrim 2y ago
  Out of curiosity, I'm looking at the JSON spec. This mildly horrifies me: "This specification allows implementations to set limits on the range and precision of numbers accepted." The spec doesn't specify a precision or range limit anywhere (just suggests that IEEE754 might be a reasonable target for interoperability, but that supports up to 64bit floats, and it looks like Oj is dropping to 32bit floats?). Python and Go don't go and change the precision of floating point numbers in their implementations, but according to the standard, they're entirely entitled to, and so is Oj. I don't see anything in https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj-modes https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj... specifying that Strict will force floating points to specific precision vs other implementations
  - byroot 2y ago
    Yes, JSON as a format is very much under specified, a lot of these sorts of things are basically implementation defined. In general libraries do what make sense in the context of their host language, or sometimes what makes sense in the context of JavaScript. For ruby/json, I consider that if something can be rountriped, from Ruby to JSON and back, it should be, which means not reducing float precision, nor integer precision, e.g. >> JSON.generate(2**128) => "340282366920938463463374607431768211456" But other libraries may consider that JSON implies JavaScript, hence the lack of big integer, so such number should be dumped as a JS decimal string or as a floating point number. > I don't see anything in [...] specifying that Strict will force floating points to specific precision vs other implementations Yes, and that's my problem with it. As you said, Oj is free to do so by the JSON spec, but I'd bet 99% of users don't know it does that, and some of them may have had data truncation in production without realizing it. So in term of matching other libraries performance, If another library is significantly faster on a given benchmark, I treat it as a bug, unless it's the result of the alternative trading what I consider correctness for speed.

byroot 2y ago

> I should point out that other Oj modes perform much better than the JSON gem Which modes are that? https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj-modes https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj... I tried: Oj.dump(obj, mode: :strict) and a few others and none seemed faster than `json 2.9.1` on the benchmarks I use. Edit: Also most of these mode simply aren't correct in my opinion: >> Oj.dump(999.9999999999999, { mode: :compat }) => "999.9999999999999" >> Oj.dump(999.9999999999999, { mode: :strict }) => "1000"

peterohler 2y ago
Using the benchmarks in the Oj test directory Oj has a slight advantage over the core json for dumping but not enough to make much difference. The comparison for Oj strict parsing compared to the core json is more substantial as 1.37 times faster. The benchmarks use a hash of mixed types included some nested elements. The callback parsers (Saj and Scp) also show a performance advantage as does the most recent Oj::Parser. As for the dumping of floats that are at the edge of precision (16 places), Oj does round to to 15 places if the last 4 of a 16 digit float is "0001" or "9999" if the float precision is not set to zero. That is intentional. If that is not the desired behavior and the Ruby conversion is preferred then setting the float precision to zero will not round. You picked the wrong options for your example. I would like to say that the core json has a come a very long way since Oj was created and is now outstanding. If the JSON gem had started out where it is now I doubt I would have bothered writing Oj.
- byroot 2y ago
  > Using the benchmarks in the Oj test directory I'm sorry, but I've looked for a while now, and I can't seem to identify the benchmark you are mentioning. I suspect it's the one John took for his benchmark suite? [0] > Oj has a slight advantage over the core json for dumping but not enough to make much difference I'd be curious to see which benchmark you are using, because on the various ones included in ruby/json, Oj is slightly slower on about all of them: https://gist.github.com/byroot/b13d78e37b5c0ac88031dff763b3bbd2 https://gist.github.com/byroot/b13d78e37b5c0ac88031dff763b3b..., except for scanning strings with lots of multi-byte characters, but I have a branch I need to finish that should fix that. > The comparison for Oj strict parsing compared to the core json is more substantial as 1.37 times faster Here too I'd be curious to see your benchmark suite because that doesn't match mine: https://gist.github.com/byroot/dd4d4391d45307a47446addeb77748dc https://gist.github.com/byroot/dd4d4391d45307a47446addeb7774... > The callback parsers (Saj and Scp) also show a performance advantage as does the most recent Oj::Parser. Yeah, callback parsing isn't something I plan to support, at least not for now. As for Oj::Parser, `ruby/json` got quite close to it, but then @tenderlove pointed to me that the API I was trying to match wasn't thread safe, hence it wasn't a fair comparison, so now I still bench against it, but with a new instance every time: https://github.com/ruby/json/pull/703 https://github.com/ruby/json/pull/703. > You picked the wrong options for you example. No, I picked them deliberately. That's the sort of behavior users don't expect and can be bitten by. As a matter of fact, I discovered this behavior because one of the benchmark payloads (canada.json) doesn't roundtrip cleanly with Oj's default mode, that's why I benchmark against the `:compat` mode. IMO truncating data for speed isn't an acceptable default config. [0] https://github.com/jhawthorn/rapidjson-ruby/blob/518818e6768c516f18cd78b095e603bf8b323bd6/benchmark/parser.rb#L36 https://github.com/jhawthorn/rapidjson-ruby/blob/518818e6768...
  - peterohler 2y ago
    The strict mode benchmarks for Oj are in the test/perf_strict.rb. Others are are in perf_*.rb. If callback parsing is not supported that's fine. Oj does support callback parsing as it allows elements in a JSON to be ignored. That save memory, GC, and performance. Your choice of course just as including callback parsers is a choice for Oj. Ok, so you picked options that you knew would fail. Again you choice but there are certainly others that would trade a slight improvement in performance to not have 16+ significant digits. It's a choice. You are certainly entitled to you opinion but that doesn't mean everyone will share them. I'm not sure what platform you are testing on but i'm sure there will be variations depending on the OS and the hardware. I tested on MacOS M1.
Twirrim 2y ago
Out of curiosity, I'm looking at the JSON spec. This mildly horrifies me: "This specification allows implementations to set limits on the range and precision of numbers accepted." The spec doesn't specify a precision or range limit anywhere (just suggests that IEEE754 might be a reasonable target for interoperability, but that supports up to 64bit floats, and it looks like Oj is dropping to 32bit floats?). Python and Go don't go and change the precision of floating point numbers in their implementations, but according to the standard, they're entirely entitled to, and so is Oj. I don't see anything in https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj-modes https://github.com/ohler55/oj/blob/develop/pages/Modes.md#oj... specifying that Strict will force floating points to specific precision vs other implementations
- byroot 2y ago
  Yes, JSON as a format is very much under specified, a lot of these sorts of things are basically implementation defined. In general libraries do what make sense in the context of their host language, or sometimes what makes sense in the context of JavaScript. For ruby/json, I consider that if something can be rountriped, from Ruby to JSON and back, it should be, which means not reducing float precision, nor integer precision, e.g. >> JSON.generate(2**128) => "340282366920938463463374607431768211456" But other libraries may consider that JSON implies JavaScript, hence the lack of big integer, so such number should be dumped as a JS decimal string or as a floating point number. > I don't see anything in [...] specifying that Strict will force floating points to specific precision vs other implementations Yes, and that's my problem with it. As you said, Oj is free to do so by the JSON spec, but I'd bet 99% of users don't know it does that, and some of them may have had data truncation in production without realizing it. So in term of matching other libraries performance, If another library is significantly faster on a given benchmark, I treat it as a bug, unless it's the result of the alternative trading what I consider correctness for speed.