5 ms·
> each thread has non-trivial overhead. Agreed. But I am not proposing a thread-per request model. The model will most likely be a fixed pool of workers, poten
by grandimam 27d ago
> each thread has non-trivial overhead.
Agreed. But I am not proposing a thread-per request model. The model will most likely be a fixed pool of workers, potentially combined with multiple processes, an M x N model.
I also agree that co-routines are cheaper than OS threads if the application itself is mostly waiting for network I/O. My question is what happens the moment the workload isn't purely I/O bound.
Right now, the solution seems to be an escape hatch where CPU-bound work is executed behind a thread-pool - see FastAPI or Starlette for example.
With free-threaded Python, I am interested in whether a framework can instead own that complexity: application code remains synchronous or async-partially (at the main thread).
> Removing GIL has nothing to do with that
It doesn't make network I/O faster, agreed. But, the more interesting consequence of removing the GIL is that with threads we can actually parallelise for CPU-bound Python code which earlier required multi-processor.
> How is it different from Flask or Django
This is probably the most important question. It's still in beta, so a lot of things can change. But simply put that Flask/Django lets one write sync code but do not provide the execution model I am describing. Concurrency is supplied at the server layer - gunicorn workers, etc.
The thesis that I am proposing is a framework runtime where concurrency and parallelism are first-class primitives. Now, that free-threaded makes threads capable of CPU parallelism can the framework own the concurrency and parallelism while application code stays async.
- theamk 26d agoCan you elaborate what does "framework runtime where concurrency and parallelism are first-class primitives" means? Because to take Flask for example - it can run the "sync code" example on your page perfectly. It even has thread-pool support, if you run it via gunicorn. And if you want to mix sync and async code, you can do this too: https://flask.palletsprojects.com/en/stable/async-await/ https://flask.palletsprojects.com/en/stable/async-await/ So what exactly is the innovation of your framework compared to that? Perhaps the easiest way to explain this would be to show some code (non-working is fine) and say: "this thing is easy in my framework, but very hard / impossible in existing ones, like Flask". I think this is what you tried to do with "what if we simply wrote" section, but you kinda failed because you can already do this anyway.