5 ms·

While I think it is always good to see progress in the performance of interpreters, ultimately it is a mistake if you have something that needs to be fast and y

While I think it is always good to see progress in the performance of interpreters, ultimately it is a mistake if you have something that needs to be fast and you implemented it entirely in an interpreted language (beyond just prototyping). You have to be prepared to factor out the key parts of the code into faster languages like C++ with bindings, if necessary. Or you can virtually do this, e.g. learn more about the standard library, make sure you are choosing things that actually are implemented natively underneath instead of in pure Python, etc. And sometimes, you find that the cause of a slowdown is far more fundamental (e.g. the entire algorithm is not good, and you see huge gains by redoing it, even if everything is still interpreted code).

rich_sasha 5y ago
> ultimately it is a mistake if you have something that needs to be fast and you implemented it entirely in an interpreted language (beyond just prototyping). Agreed. Equally the faster the interpreted language, the less you need to offload, because baseline speed is good enough. Faster is rarely a bad thing.
xigoi 5y ago
Or you could just use a good compiled language for everything.
pizza234 5y ago
> You have to be prepared to factor out the key parts of the code into faster languages like C++ with bindings, if necessary Stripe had the innovative idea of compiling ahead of time parts of the code (taking advantage of type annotations). I think this is a very interesting approach that may make usage of faster language unnecessary in a certain amount of cases.
- staticassertion 5y ago
  Dropbox has some projects around this. I think the idea is totally silly tbh and it flies in the face of Python's type annotations being just that - annotations. The generated code is pretty hilarious since you end up with HashMaps and dynamic dispatch everywhere. Much better gains can be made by simply not using Python.
  - pjmlp 5y ago
    I learned that lesson with Tcl back in my first startup experience. Sure the language is great, and it is very easy to write extensions in C, however as the performance pressure keeps increasing, in the end it is a C application where Tcl was reduced to a configuration/orchestration language. Since then, if it doesn't have a JIT or AOT compiler in the box I am not interested, unless I am obliged to do so by higher levels.
pthread_t 5y ago
I've found that depending on that "key part", you can actually offload the task to another process (written in C/C++/Rust) via RPC. The end result was a significant performance improvement.
- mattgreenrocks 5y ago
  Using something like PyO3 for Rust/Python integration really helps with this, and you don't need the conceptual overhead of RPC.
  - joshuamorton 5y ago
    Interesting, I find that the conceptual overhead of pyo3 (or cytpes) is often higher than making a local rpc
- makecheck 5y ago
  I find that subprocesses are primarily valuable for stability and security, e.g. dealing with something that might crash (killing only the subprocess) or something that requires different privileges than the main process. And yet, the communication to (and possibly from) the child incurs a cost that cannot be ignored, more than what a simple wrapper to another language would have.
- IshKebab 5y ago
  I work on a system like that and it just means you have to deal with a load of complicated FFI stuff and you end up with a C++ program that happens to have some slow parts awkwardly implemented in buggy Python.
boxed 5y ago
When you've done all that you're still left with performance wins you could get that are in the interpreter. I've worked on a system like that where it was just nothing to optimize really (anymore). It was just lots of code and there was no hot path to optimize.
trulyme 5y ago
>... ultimately it is a mistake if you have something that needs to be fast and you implemented it entirely in an interpreted language (beyond just prototyping). Yes. However, this is exactly where Python (imho) shines. You get a convenient interpreted language which seemlessly interacts with highly optimized native libraries like numpy, scipy, xarray,... and of course ML frameworks (where impact of base language speed is negligable compared to the time it takes to train a model).
- throwaway894345 5y ago
  I’ve been using Python for 15 years, and while sometimes it’s a good fit for “write the fast parts in C!” type workloads, these seem to be very rare. Most of the time you spend more time marshaling back and forth between Python data structures and C data structures and you end up losing more performance (and maintainability and build system complexity and so on). If you have an application that might ever have a hot path, I heartily do not recommend python.
  - trulyme 5y ago
    It probably depends on the type of apps one is writing. I am not advocating writing your own libs (though that is completely feasible too) but just using the existing ones. When one needs to convert from/to Python structures within the hot path, this of course doesn't perform well... However in most cases I worked on this was not needed - convert input to numpy / xarray /..., perform all calculations, get the result. So with about the same amount of experience, I heartily recommend python. :)
    - throwaway894345 5y ago
      The problem is that you can't predict from the outset that all of your performance problems will be a good fit for Numpy or whatever, and if they aren't the options you have are very expensive in terms of engineering time, maintainability, etc. And it's not just performance--dependency management is still a significant problem, as is deployment in many cases. Honestly, I switched to Go. It's like a fast, statically typed, statically compiled Python. And by "fast", I mean 100-1000x faster than CPython. The package manager resolves dependencies in an instant (compared with pipenv that would take 30 minutes just to update a lockfile for a small project). Everything compiles to a static binary so you can build internal tools without requiring end users to set up a virtualenv/etc. I've rewritten real world Python programs that would be distributed as a 250MB zip file or a multi-GB docker image and converted them into Go programs which would distribute as a 2.5MB binary or 2.5MB docker image. Docker image builds went from 45 minutes (after weeks of painstakingly optimizing Dockerfiles) to ~2 minutes with a straightforward Dockerfile. The ramifications of a fast iteration loop are also hard to overstate--this was a major source of pain that virtually disappeared when the application was ported to Go. On AWS ECS/Fargate and GKE, container cold start times went from 30-50 minutes to ~30s (not sure why pulling from ECR/GCR and then starting the container took so long on these platforms--I certainly wouldn't expect a few GB to take half an hour to pull over a datacenter network). The performance also improved tremendously--in one case we traded so much maintainability in order to get a Python/Pandas system to complete requests within 60s and a naive Go rewrite would complete the same requests in 2 seconds (CPU intense parallel-friendly workload). We also looked at Dask, Polars, Spark, etc. With Python, we ran into significant Docker For Mac performance problems (the Docker VM would use all available CPU to marshal filesystem events to/from the VM, grinding the app to a halt and chewing through your battery) with Go there's no need to mount a source code filesystem to mount in the first place--you just rebuild on change (either the whole image or just rebuild the binary and `docker cp` it onto the running container). Of course, I'm sure there are other language environments that also have benefits over Python. I've just found Go to be the best I've tried by a pretty wide margin. Frankly, Python is just littered with pitfalls--you just sort of find yourself painted into these corners unexpectedly and you waste a bunch of time trying out all of these tools, frameworks, and libraries that purport to solve your problem but which introduce their own novel failure modes for nontrivial applications (e.g., Pipenv promises to solve dependency management but every lockfile operation takes half an hour, Cython promises to improve performance but it adds considerable complexity to your build tooling, Mypy promises to fix typing but it can never find the type annotations for dependencies, Pypy promises to improve performance but a ton of important ecosystem packages are unsupported, etc). Python development just feels like wandering from one tarpit to another, while in Go I just get things done.
matteotom 5y ago
In my experience, the important part is sometimes that _faster_ means _cheaper_. In many cases, 20% faster code means you need almost 20% fewer CPUs, which really adds up at scale.

rich_sasha 5y ago

> ultimately it is a mistake if you have something that needs to be fast and you implemented it entirely in an interpreted language (beyond just prototyping). Agreed. Equally the faster the interpreted language, the less you need to offload, because baseline speed is good enough. Faster is rarely a bad thing.

xigoi 5y ago

Or you could just use a good compiled language for everything.

pizza234 5y ago

> You have to be prepared to factor out the key parts of the code into faster languages like C++ with bindings, if necessary Stripe had the innovative idea of compiling ahead of time parts of the code (taking advantage of type annotations). I think this is a very interesting approach that may make usage of faster language unnecessary in a certain amount of cases.

staticassertion 5y ago
Dropbox has some projects around this. I think the idea is totally silly tbh and it flies in the face of Python's type annotations being just that - annotations. The generated code is pretty hilarious since you end up with HashMaps and dynamic dispatch everywhere. Much better gains can be made by simply not using Python.
- pjmlp 5y ago
  I learned that lesson with Tcl back in my first startup experience. Sure the language is great, and it is very easy to write extensions in C, however as the performance pressure keeps increasing, in the end it is a C application where Tcl was reduced to a configuration/orchestration language. Since then, if it doesn't have a JIT or AOT compiler in the box I am not interested, unless I am obliged to do so by higher levels.

pthread_t 5y ago

I've found that depending on that "key part", you can actually offload the task to another process (written in C/C++/Rust) via RPC. The end result was a significant performance improvement.

mattgreenrocks 5y ago
Using something like PyO3 for Rust/Python integration really helps with this, and you don't need the conceptual overhead of RPC.
- joshuamorton 5y ago
  Interesting, I find that the conceptual overhead of pyo3 (or cytpes) is often higher than making a local rpc
makecheck 5y ago
I find that subprocesses are primarily valuable for stability and security, e.g. dealing with something that might crash (killing only the subprocess) or something that requires different privileges than the main process. And yet, the communication to (and possibly from) the child incurs a cost that cannot be ignored, more than what a simple wrapper to another language would have.
IshKebab 5y ago
I work on a system like that and it just means you have to deal with a load of complicated FFI stuff and you end up with a C++ program that happens to have some slow parts awkwardly implemented in buggy Python.

boxed 5y ago

When you've done all that you're still left with performance wins you could get that are in the interpreter. I've worked on a system like that where it was just nothing to optimize really (anymore). It was just lots of code and there was no hot path to optimize.

trulyme 5y ago

>... ultimately it is a mistake if you have something that needs to be fast and you implemented it entirely in an interpreted language (beyond just prototyping). Yes. However, this is exactly where Python (imho) shines. You get a convenient interpreted language which seemlessly interacts with highly optimized native libraries like numpy, scipy, xarray,... and of course ML frameworks (where impact of base language speed is negligable compared to the time it takes to train a model).

throwaway894345 5y ago
I’ve been using Python for 15 years, and while sometimes it’s a good fit for “write the fast parts in C!” type workloads, these seem to be very rare. Most of the time you spend more time marshaling back and forth between Python data structures and C data structures and you end up losing more performance (and maintainability and build system complexity and so on). If you have an application that might ever have a hot path, I heartily do not recommend python.
- trulyme 5y ago
  It probably depends on the type of apps one is writing. I am not advocating writing your own libs (though that is completely feasible too) but just using the existing ones. When one needs to convert from/to Python structures within the hot path, this of course doesn't perform well... However in most cases I worked on this was not needed - convert input to numpy / xarray /..., perform all calculations, get the result. So with about the same amount of experience, I heartily recommend python. :)
  - throwaway894345 5y ago
    The problem is that you can't predict from the outset that all of your performance problems will be a good fit for Numpy or whatever, and if they aren't the options you have are very expensive in terms of engineering time, maintainability, etc. And it's not just performance--dependency management is still a significant problem, as is deployment in many cases. Honestly, I switched to Go. It's like a fast, statically typed, statically compiled Python. And by "fast", I mean 100-1000x faster than CPython. The package manager resolves dependencies in an instant (compared with pipenv that would take 30 minutes just to update a lockfile for a small project). Everything compiles to a static binary so you can build internal tools without requiring end users to set up a virtualenv/etc. I've rewritten real world Python programs that would be distributed as a 250MB zip file or a multi-GB docker image and converted them into Go programs which would distribute as a 2.5MB binary or 2.5MB docker image. Docker image builds went from 45 minutes (after weeks of painstakingly optimizing Dockerfiles) to ~2 minutes with a straightforward Dockerfile. The ramifications of a fast iteration loop are also hard to overstate--this was a major source of pain that virtually disappeared when the application was ported to Go. On AWS ECS/Fargate and GKE, container cold start times went from 30-50 minutes to ~30s (not sure why pulling from ECR/GCR and then starting the container took so long on these platforms--I certainly wouldn't expect a few GB to take half an hour to pull over a datacenter network). The performance also improved tremendously--in one case we traded so much maintainability in order to get a Python/Pandas system to complete requests within 60s and a naive Go rewrite would complete the same requests in 2 seconds (CPU intense parallel-friendly workload). We also looked at Dask, Polars, Spark, etc. With Python, we ran into significant Docker For Mac performance problems (the Docker VM would use all available CPU to marshal filesystem events to/from the VM, grinding the app to a halt and chewing through your battery) with Go there's no need to mount a source code filesystem to mount in the first place--you just rebuild on change (either the whole image or just rebuild the binary and `docker cp` it onto the running container). Of course, I'm sure there are other language environments that also have benefits over Python. I've just found Go to be the best I've tried by a pretty wide margin. Frankly, Python is just littered with pitfalls--you just sort of find yourself painted into these corners unexpectedly and you waste a bunch of time trying out all of these tools, frameworks, and libraries that purport to solve your problem but which introduce their own novel failure modes for nontrivial applications (e.g., Pipenv promises to solve dependency management but every lockfile operation takes half an hour, Cython promises to improve performance but it adds considerable complexity to your build tooling, Mypy promises to fix typing but it can never find the type annotations for dependencies, Pypy promises to improve performance but a ton of important ecosystem packages are unsupported, etc). Python development just feels like wandering from one tarpit to another, while in Go I just get things done.

matteotom 5y ago

In my experience, the important part is sometimes that _faster_ means _cheaper_. In many cases, 20% faster code means you need almost 20% fewer CPUs, which really adds up at scale.