5 ms·

Does Python still need to be async-first?

2 points by grandimam 27d ago

theamk 26d ago
Author seems to miss the the primary reason people build network apps with async: resource usage. If you don't have co-routines, you need to have something else. That "something else" is most commonly threads, and each thread has non-trivial overhead, in scheduling and memory. A coroutine is much more memory efficient. Removing GIL has nothing to do with that, coroutines are even worse re GIL problems. And of course, there are plenty of thread-based frameworks for Python, already - Flask and Django are most well known examples, but there are tons of others. So my first question for author would be: "how is this different from Flask"?
- grandimam 26d ago
  > each thread has non-trivial overhead. Agreed. But I am not proposing a thread-per request model. The model will most likely be a fixed pool of workers, potentially combined with multiple processes, an M x N model. I also agree that co-routines are cheaper than OS threads if the application itself is mostly waiting for network I/O. My question is what happens the moment the workload isn't purely I/O bound. Right now, the solution seems to be an escape hatch where CPU-bound work is executed behind a thread-pool - see FastAPI or Starlette for example. With free-threaded Python, I am interested in whether a framework can instead own that complexity: application code remains synchronous or async-partially (at the main thread). > Removing GIL has nothing to do with that It doesn't make network I/O faster, agreed. But, the more interesting consequence of removing the GIL is that with threads we can actually parallelise for CPU-bound Python code which earlier required multi-processor. > How is it different from Flask or Django This is probably the most important question. It's still in beta, so a lot of things can change. But simply put that Flask/Django lets one write sync code but do not provide the execution model I am describing. Concurrency is supplied at the server layer - gunicorn workers, etc. The thesis that I am proposing is a framework runtime where concurrency and parallelism are first-class primitives. Now, that free-threaded makes threads capable of CPU parallelism can the framework own the concurrency and parallelism while application code stays async.
  - theamk 26d ago
    Can you elaborate what does "framework runtime where concurrency and parallelism are first-class primitives" means? Because to take Flask for example - it can run the "sync code" example on your page perfectly. It even has thread-pool support, if you run it via gunicorn. And if you want to mix sync and async code, you can do this too: https://flask.palletsprojects.com/en/stable/async-await/ https://flask.palletsprojects.com/en/stable/async-await/ So what exactly is the innovation of your framework compared to that? Perhaps the easiest way to explain this would be to show some code (non-working is fine) and say: "this thing is easy in my framework, but very hard / impossible in existing ones, like Flask". I think this is what you tried to do with "what if we simply wrote" section, but you kinda failed because you can already do this anyway.

2 points by grandimam 27d ago

Author seems to miss the the primary reason people build network apps with async: resource usage. If you don't have co-routines, you need to have something else. That "something else" is most commonly threads, and each thread has non-trivial overhead, in scheduling and memory. A coroutine is much more memory efficient. Removing GIL has nothing to do with that, coroutines are even worse re GIL problems. And of course, there are plenty of thread-based frameworks for Python, already - Flask and Django are most well known examples, but there are tons of others. So my first question for author would be: "how is this different from Flask"?

grandimam 26d ago
> each thread has non-trivial overhead. Agreed. But I am not proposing a thread-per request model. The model will most likely be a fixed pool of workers, potentially combined with multiple processes, an M x N model. I also agree that co-routines are cheaper than OS threads if the application itself is mostly waiting for network I/O. My question is what happens the moment the workload isn't purely I/O bound. Right now, the solution seems to be an escape hatch where CPU-bound work is executed behind a thread-pool - see FastAPI or Starlette for example. With free-threaded Python, I am interested in whether a framework can instead own that complexity: application code remains synchronous or async-partially (at the main thread). > Removing GIL has nothing to do with that It doesn't make network I/O faster, agreed. But, the more interesting consequence of removing the GIL is that with threads we can actually parallelise for CPU-bound Python code which earlier required multi-processor. > How is it different from Flask or Django This is probably the most important question. It's still in beta, so a lot of things can change. But simply put that Flask/Django lets one write sync code but do not provide the execution model I am describing. Concurrency is supplied at the server layer - gunicorn workers, etc. The thesis that I am proposing is a framework runtime where concurrency and parallelism are first-class primitives. Now, that free-threaded makes threads capable of CPU parallelism can the framework own the concurrency and parallelism while application code stays async.
- theamk 26d ago
  Can you elaborate what does "framework runtime where concurrency and parallelism are first-class primitives" means? Because to take Flask for example - it can run the "sync code" example on your page perfectly. It even has thread-pool support, if you run it via gunicorn. And if you want to mix sync and async code, you can do this too: https://flask.palletsprojects.com/en/stable/async-await/ https://flask.palletsprojects.com/en/stable/async-await/ So what exactly is the innovation of your framework compared to that? Perhaps the easiest way to explain this would be to show some code (non-working is fine) and say: "this thing is easy in my framework, but very hard / impossible in existing ones, like Flask". I think this is what you tried to do with "what if we simply wrote" section, but you kinda failed because you can already do this anyway.