itamarst

6 ms·

1.
by itamarst 2mo ago
When this was posted to lobsters someone shared this relevant link: https://curiouscoding.nl/posts/static-search-tree/
2.
by itamarst 2mo ago
The bucket boundaries are often chosen from a random sample of the data, if the input data is very large. Sorting is O(nlogn), but using binary search per value to assign a bucket is O(n), plus the cost of creating the buckets on a sample.
3.
by itamarst 2mo ago
Better guesses reduce the number of guesses, so there will be less branch misprediction, but there will still be mispredictions for each remaining branch. So I would guess branchless interpolation search would still help. In practice becaus
4.
Testing the compiler optimizations your code relies on(pythonspeed.com)
1 points by itamarst 1y ago|0 comments
5.
500× faster: Four different ways to speed up your code(pythonspeed.com)
3 points by itamarst 1y ago|0 comments
6.
by itamarst 1y ago
That's great! Would also be cool (separately from Pydantic use case) to add jiter backend to ijson.
7.
by itamarst 1y ago
https://remarkjs.com/
8.
by itamarst 1y ago
I talk about this more explicitly in the PyCon talk ( https://pythonspeed.com/pycon2025/slides/ - video soon) though that's not specifically about Pydantic, but basically: 1. Inefficient parser implementation.
9.
by itamarst 1y ago
Once you switch to ijson it will not save any memory, no, because ijson essentially uses zero memory for the parsing. You're just left with the in-memory representation.
10.
by itamarst 1y ago
msgspec is much more memory efficient out of the box, yes. Also quite fast.
11.
by itamarst 1y ago
You can't add extra attributes that weren't part of the original dataclass definition: >>> from dataclasses import dataclass >>> @dataclass ... class C: pass ... >>> C().x = 1 >>>
12.
by itamarst 1y ago
For my PyCon 2025 talk I did this. Video isn't up yet, but slides are here: https://pythonspeed.com/pycon2025/slides/ The linked-from-original-article ijson article was the inspiration for the talk: https:&#
13.
by itamarst 1y ago
One key question in these sort of things is how free() works: it is given a pointer, and it has to decide whether this was sampled or not, with _minimum_ effort. Poireau does this, IIRC, by putting the pointers it sampled in a different mem
14.
Loading Pydantic models from JSON without running out of memory(pythonspeed.com)
134 points by itamarst 1y ago|45 comments
15.
The surprising way to save memory with BytesIO(pythonspeed.com)
4 points by itamarst 2y ago|0 comments
16.
Faster pip installs: caching, bytecode compilation, and uv(pythonspeed.com)
2 points by itamarst 2y ago|0 comments
17.
Catching memory leaks with your test suite(pythonspeed.com)
3 points by itamarst 2y ago|0 comments
18.
Reducing CO₂ emissions with faster software(pythonspeed.com)
3 points by itamarst 2y ago|0 comments
19.
Using portable SIMD in stable Rust(pythonspeed.com)
1 points by itamarst 2y ago|0 comments
20.
by itamarst 2y ago
It's somewhat domain specific. Pure Python libraries have easier time supporting new releases than libraries that rely on C APIs, and even slower are those that deal with less stable implementation details like bytecode (e.g. Numba). B
21.
Beyond multi-core parallelism: faster Mandelbrot with SIMD(pythonspeed.com)
4 points by itamarst 2y ago|0 comments
22.
Let's build and optimize a Rust extension for Python(pythonspeed.com)
4 points by itamarst 2y ago|0 comments
23.
by itamarst 2y ago
It's much faster! There's been significant performance improvements since 3.8.
24.
It's time to stop using Python 3.8(pythonspeed.com)
5 points by itamarst 2y ago|5 comments
25.
Running 15× faster with a situation-specific algorithm(pythonspeed.com)
4 points by itamarst 2y ago|0 comments
26.
by itamarst 3y ago
There's overhead in transferring data from CPU to GPU and back. I'm not sure how this works with internal GPUs, though, insofar as RAM is shared. In general, though, as I understand it (not a GPU programmer) you want to pass data
27.
Jevons Paradox doesn't always apply to software(pythonspeed.com)
2 points by itamarst 3y ago|0 comments
28.
Not just Nvidia: GPU programming that runs everywhere(pythonspeed.com)
1 points by itamarst 3y ago|0 comments
29.
by itamarst 3y ago
You can use Numba to speed up some Pandas calculations: https://pandas.pydata.org/docs/user_guide/enhancingperf.html...
30.
Profiling your Numba code(pythonspeed.com)
67 points by itamarst 3y ago|23 comments

More ›