jammycrisp

5 ms·

1.
Faster, more memory-efficient Python JSON parsing with msgspec(pythonspeed.com)
2 points by jammycrisp 3y ago|0 comments
2.
by jammycrisp 3y ago
Yep! Both msgspec ( https://jcristharif.com/msgspec/supported-types.html#datacla... ) and orjson support encoding dataclasses to JSON natively.
3.
by jammycrisp 3y ago
> we measure the number of instructions and memory/cache accesses through CPU instrumentation performed with Valgrind. This approach gives repeatable and consistent results that couldn’t be obtained with a time based statistical app
4.
by jammycrisp 3y ago
starlite was the original name, it was recently renamed to litestar due to comments about how easily confused "starlette" and "starlite" are.
5.
by jammycrisp 3y ago
+1 for litestar[1]. The higher bus-factor is nice, and I like that they're working to embrace a wider set of technologies than just pydantic. The framework currently lets you model objects using msgspec[2] (they actually use msgspec fo
6.
by jammycrisp 3y ago
If you like cattrs, you _might_ be interested in trying out my msgspec library [1]. It works out-of-the-box with attrs objects (as well as its own faster `Struct` types), while being ~10-15x faster than cattrs for encoding/decoding&#x2
7.
by jammycrisp 3y ago
> Maybe it was very slow before That is at least partly the case. I maintain msgspec[1], another Python JSON validation library. Pydantic V1 was ~100x slower at encoding/decoding/validating JSON than msgspec, which was more a t
8.
by jammycrisp 3y ago
While it's definitely much faster than pydantic V1 (which is a huge accomplishment!), it's still not exactly what I'd call "fast". I maintain msgspec ( https://github.com/jcrist/msgspec ), a seri
9.
Show HN: Msgspec, a fast serialization/validation library for Python(github.com)
3 points by jammycrisp 3y ago|1 comments
10.
by jammycrisp 3y ago
Thanks, glad you like it!
11.
by jammycrisp 3y ago
While I agree that there are ways to write a faster validation library in python, there are also benefits to moving the logic to native code. msgspec[1] is another parsing/validation library, written in C. It's on average 50-80x f
12.
by jammycrisp 3y ago
It looks like pydantic-core is distributing musllinux wheels, which should work fine on alpine. Fwiw tooling like cibuildwheel makes building and publishing wheels for all the common platforms fairly straightforward now.
13.
by jammycrisp 3y ago
Are there any necessary features that you've found missing in msgspec? One of the design goals for msgspec (besides much higher performance) was simpler usage. Fewer concepts to wrap your head around, fewer config options to learn abou
14.
by jammycrisp 4y ago
If you're primarily targeting Python as an application layer, you may also want to check out my msgspec library[1]. All the perf benefits of e.g. yyjson, but with schema validation like pydantic. It regularly benchmarks[2] as the faste
15.
by jammycrisp 4y ago
Pytest has an equally deprecating option for a different "use case": disable_test_id_escaping_and_forfeit_all_rights_to_community_support = True https://docs.pytest.org/en/6.2.x/parametrize.html
16.
by jammycrisp 4y ago
Integrating `dataclass` more into the language builtins might be nice, if only that it may allow/encourage a more native and performant implementation. Using a dataclass right now results in slightly slower class operations than handwr
17.
by jammycrisp 4y ago
You might be interested in checking out Ibis ( https://ibis-project.org/ ). It provides a dataframe-like API, abstracting over many common execution engines (duckdb, postgres, bigquery, spark, ...). Ibis wrapping duckdb has p
18.
by jammycrisp 4y ago
For creating images without docker from conda/mamba environments, there's also the existing `conda-docker` tool https://github.com/conda-incubator/conda-docker .
19.
by jammycrisp 4y ago
There's also https://github.com/llllllllll/phorth , an implementation of forth that compiles to cpython bytecode
20.
by jammycrisp 4y ago
For my own "fast" projects[1] I've taken to providing benchmarks, but adding a big 'ol caveat at the top describing ways in which the benchmark may not reflect reality. Some people really want to see benchmarks! Others a
21.
Ibis-datasette: query datasettes via a Python dataframe API(jcristharif.com)
1 points by jammycrisp 4y ago|0 comments
22.
Faster, more memory-efficient Python JSON parsing with msgspec(pythonspeed.com)
2 points by jammycrisp 4y ago|0 comments
23.
Show HN: Msgspec – a fast JSON schema library for Python(jcristharif.com)
2 points by jammycrisp 4y ago|0 comments
24.
by jammycrisp 4y ago
GitHub issue: https://github.com/psf/requests/issues/6140
25.
by jammycrisp 4y ago
> I should mention that spyql leverages orjson, which has a considerable impact on performance Even with orjson, you're still paying the cost of creating a new PyObject for every node in the JSON blob. orjson is well engineered (as
26.
Show HN: Msgspec – a fast JSON/MessagePack library for Python(jcristharif.com)
2 points by jammycrisp 5y ago|0 comments
27.
by jammycrisp 5y ago
No use in changing things if it's working for you, but you might be interested in trying out msgspec ( https://jcristharif.com/msgspec/ ) in tino instead of using msgpack-python. The msgpack encoder/decoder is
28.
by jammycrisp 5y ago
Yes, but it's also less flexible. Tradeoffs.
29.
by jammycrisp 5y ago
Y'all may be interested in a fast dataclass-like library I maintain called msgspec ( https://jcristharif.com/msgspec/ ) that provides many of the benefits of dataclasses (mutable, type declarations), but with speedy
30.
by jammycrisp 5y ago
The author used to work on Eve.

More ›