7 ms·

Async Django in Production

50 points by jonathan-adly 2y ago

singularity2001 2y ago
When will it be common knowledge that introducing async function coloring (on the callee side, instead of just 'go' on the call side) was the biggest mistake of the decade?
- gnlampx 2y ago
  This will happen when someone writes a new green thread framework and convinces van Rossum to preside over the corporate-sponsored integration into Python. Four releases later async will be marketed as "this has been a mistake" and the green threads will be the best thing ever. Thousands of blog posts will be written and everyone has to rewrite their code bases. Repeat for goroutines etc.
  - simonw 2y ago
    As Python grows GIL-free threading and machines continue to grow more cores do you think we'll be able to skip green threads and use regular threads instead?
    - deleted 2y ago
      [deleted]
- the__alchemist 2y ago
  I feel like I'm a loner; the odd man out regarding this in rust. I can no longer relate to my (online/OSS) peers, nor use the same libraries they do. I almost wish sync and async rust were separate languages. The web and embedded communities are almost fully committed. I stopped participating in the rust OSS embedded community for this reason. I haven't tried Async Django, (But use normal Django on a few work and hobby project) and am hesitant based on my experience in rust, and not finding a fault I think coloring my Python/Django code would be advantageous to do.
- petters 2y ago
  Colored functions can be nice. It makes the type system give a hint that a function could take a long time. You can then e.g. avoid holding locks when awaiting. But there is a big downside, that's true.
- Eikon 2y ago
  I feel I may be an outlier here, but I love async functions (I mostly use Rust). I intuitively understand what will block and what won't, and can generally gauge the performance impact. I believe the few downsides are far outweighed by the massive benefits. Async functions allow me to build massively parallel and concurrent systems with ease - it's beautiful. In comparison, I'm not as fond of Go's approach to concurrency, which feels less elegant to me.
  - UltraSane 2y ago
    I found learning async Python to be very painful for months until I gained an intuitive mental model of the async code flow and then it all just started to "click" for me. I just think of async as parallelized waiting.
  - dangsux 2y ago
    [dead]
- 10000truths 2y ago
  Function coloring is only a symptom of a larger problem. The problem is that libraries implementing databases and network protocols are designed in such a way that they tightly couple themselves to a particular type of I/O dispatch. Ideally, libraries wouldn't be dispatching any I/O at all! Instead, they'd have a sans-I/O [0] implementation, where the caller creates a state machine object, feeds it a byte buffer, gets a byte buffer, and then the caller has to handle reading/writing those buffers from/to a socket or disk. A well-designed sans-IO library wouldn't care whether you're using it with sync I/O, or async I/O, or with gevent, or with a homegrown event loop built on epoll or io_uring or kqueue or IOCP. [0] https://sans-io.readthedocs.io/ https://sans-io.readthedocs.io/
  - surajrmal 2y ago
    It's not always that simple. For a complicated self contained state machine this might work, but what about a situation where you are coordinating interactions with multiple distinct actors? You need a primitive to wait. At that point you need to pick a threading model.
    - 10000truths 2y ago
      State machines compose - the state of each actor is a part of your overall application state. So you'd create a giant state machine that contains a smaller state machine for each actor. Waits and timeouts can be handled by encapsulating time into your state. Then, you can feed your state machine with "X amount of time has elapsed" messages to inform it of the passage of time, and your state machine can emit something like a "sessions 123, 456 timed out" message as a response, and you can act on that by closing the sockets associated with those sessions.
- surajrmal 2y ago
  In languages where there is already considerable overhead I agree with you, but go style coroutines are not really a competitive option for some use cases. The alternative is callbacks or something else which is a lightweight abstraction on top of an event loop.
- jfasydfw 2y ago
  gevent was created over a decade ago to solve this. python users only have themselves to blame for ignoring it.
  - westurner 2y ago
    From "Asyncio, twisted, tornado, gevent walk into a bar" (2023) https://news.ycombinator.com/item?id=37227567 https://news.ycombinator.com/item?id=37227567 : > IIRC the history of the async things in TLA in order: Twisted (callbacks), Eventlet (for Second Life by Linden Labs), tornado, gevent; gunicorn, Python 3.5+ asyncio, [uvicorn,] Async/await > History: https://en.wikipedia.org/wiki/Async/await https://en.wikipedia.org/wiki/Async/await
jonathan-adly 2y ago
We have traditionally used Django in all our projects. We believe it is one of the most underrated, beautifully designed, rock solid framework out there. However, if we are to be honest, the history of async usage in Django wasn't very impressive. You could argue that for most products, you don’t really need async. It was just an extra layer of complexity without any significant practical benefit. Over the last couple of years, AI use-cases have changed that perception. Many AI products have calling external APIs over the network as their bottleneck. This makes the complexity from async Python worth considering. FastAPI with its intuitive async usage and simplicity have risen to be the default API/web layer for AI projects. I wrote about using async Django in a relatively complex AI open source project here: https://jonathanadly.com/is-async-django-ready-for-prime-time https://jonathanadly.com/is-async-django-ready-for-prime-tim... tldr: Async django is ready! there is a couple of gotcha's here and there, but there should be no performance loss when using async Django instead of FastAPI for the same tasks. Django's built-in features greatly simplify and enhance the developer experience. So - go ahead and use async Django in your next project. It should be a lot smoother that it was a year or even six months ago.
- fulafel 2y ago
  (Disclaimer: I haven't used Django in a long time) Can you expand more on why these AI cases make the complexity tradeoff different? I'd imagine think waiting on a 3rd party LLM API call would be computationally very inexpensive compared to what's going on at the business end of that API call. Further lowering the cost, Django is usually configured to use multiple threads and/or processes so that this blocking call won't keep a CPU idle, no?
  - flowingfocus 2y ago
    The explanation is in the article. Tldr is for sync functions, the CPU is blocked, with async functions, once the await statement is reached, other stuff can be handled in between
    - fulafel 2y ago
      Indeed it says "It enhances performance in areas where tasks are waiting for IO to complete by allowing the CPU to handle other tasks in the meantime". To restate my comment: I argued (1) this CPU cost would be very marginal compared to the LLM API compute cost, and (2) the CPU blocking claim doesn't really hold, due to the wonders of threads and processes.
andrewstuart 2y ago
Async Django is a bit of a puzzle …. who is it for? People who like synchronous Python can use Django. People who like asynch Python can use Starlette - the async web server also written by the guy who wrote Django. It’s not clear why Django needs to be async, especially when I get the sense there’s developers who like async and developers who prefer synch. It’s a natural fit for Django to fulfill the sync demand and Starlette to fulfill the async. They’re both good.
- senko 2y ago
  Starlette and Django are wildly different. Django has a batteries-includes approach, benefits from tighter integration of orm, auth, form handling, etc, and has a huge 3rd party ecosystem. First-class async support in Django allows Django users to avoid jumping through hoops (celery, channels, ...) for longer-running requests, something especially noticable if you're calling any kind of AI service.
  - globular-toast 2y ago
    How much of that is relevant if you're going to be using django-ninja (pydantic) and the whole app has to be async, though? Django is fine for writing a thin CRUD layer around a database. It makes the easy stuff easy. But doesn't seem to help much for the hard stuff and often actively hinders it. Really the main reason for Django is its ORM and migrations. It's basically the other Python ORM (next to SQLAlchemy) but, unlike SQLAlchemy, it's not designed to be used standalone. In my experience I find Django (and active record ORMs in general) easier for people to get started with, but massively limiting in long run for complex domains.
    - senko 2y ago
      > if you're going to be using django-ninja (pydantic) This assumes that people don't do multi-page apps or sites any more, which ... isn't true. And I believe django-ninja replaces forms/serialization/deserialization and routing, while nicely integrating with everything else. > Django is fine for writing a thin CRUD layer around a database. In my dozen or so years with Django, I confess I did more than a few thin CRUD layers around a database. But also worked on billing systems for telecoms, insurance provider API services, live/on demand audio/video streaming services, a bunch of business process apps, AI codegen tools, and other web apps and API backends that were way more than thin CRUD layers around databases. Django was rarely a hindrance. In fact, Python being sync-only (or Django not supporting async) was usually more of a hindrance that anything Django specific. > In my experience I find Django (and active record ORMs in general) easier for people to get started with, but massively limiting in long run for complex domains. In my exprience the only situations where Django's ORM doesn't help much is when you have a lot of business logic in your database (views, stored procedures), or the database is organized in a way that's not Django's preffered way. Still works, mind you, just not as great a experience. However, the vast majority of projects I've encountered have none of those. Otherwise, I've found its ORM quite powerful, and easy to drop down to raw() in cases where you really need it (which was maybe 1% on the projects I've worked).
pacifika 2y ago
Learned quite a bit thanks!
olgeni 2y ago
I was using Daphne and then found out that it read the whole files in a POST before passing it along - needless to say I am no longer using Daphne \o/
beestripes 2y ago
Nearly all of the work done on Django ninja is by someone who lists their location on GitHub as Kharkiv. I hope that’s not current.

50 points by jonathan-adly 2y ago

singularity2001 2y ago

When will it be common knowledge that introducing async function coloring (on the callee side, instead of just 'go' on the call side) was the biggest mistake of the decade?

gnlampx 2y ago
This will happen when someone writes a new green thread framework and convinces van Rossum to preside over the corporate-sponsored integration into Python. Four releases later async will be marketed as "this has been a mistake" and the green threads will be the best thing ever. Thousands of blog posts will be written and everyone has to rewrite their code bases. Repeat for goroutines etc.
- simonw 2y ago
  As Python grows GIL-free threading and machines continue to grow more cores do you think we'll be able to skip green threads and use regular threads instead?
  - deleted 2y ago
    [deleted]
the__alchemist 2y ago
I feel like I'm a loner; the odd man out regarding this in rust. I can no longer relate to my (online/OSS) peers, nor use the same libraries they do. I almost wish sync and async rust were separate languages. The web and embedded communities are almost fully committed. I stopped participating in the rust OSS embedded community for this reason. I haven't tried Async Django, (But use normal Django on a few work and hobby project) and am hesitant based on my experience in rust, and not finding a fault I think coloring my Python/Django code would be advantageous to do.
petters 2y ago
Colored functions can be nice. It makes the type system give a hint that a function could take a long time. You can then e.g. avoid holding locks when awaiting. But there is a big downside, that's true.
Eikon 2y ago
I feel I may be an outlier here, but I love async functions (I mostly use Rust). I intuitively understand what will block and what won't, and can generally gauge the performance impact. I believe the few downsides are far outweighed by the massive benefits. Async functions allow me to build massively parallel and concurrent systems with ease - it's beautiful. In comparison, I'm not as fond of Go's approach to concurrency, which feels less elegant to me.
- UltraSane 2y ago
  I found learning async Python to be very painful for months until I gained an intuitive mental model of the async code flow and then it all just started to "click" for me. I just think of async as parallelized waiting.
- dangsux 2y ago
  [dead]
10000truths 2y ago
Function coloring is only a symptom of a larger problem. The problem is that libraries implementing databases and network protocols are designed in such a way that they tightly couple themselves to a particular type of I/O dispatch. Ideally, libraries wouldn't be dispatching any I/O at all! Instead, they'd have a sans-I/O [0] implementation, where the caller creates a state machine object, feeds it a byte buffer, gets a byte buffer, and then the caller has to handle reading/writing those buffers from/to a socket or disk. A well-designed sans-IO library wouldn't care whether you're using it with sync I/O, or async I/O, or with gevent, or with a homegrown event loop built on epoll or io_uring or kqueue or IOCP. [0] https://sans-io.readthedocs.io/ https://sans-io.readthedocs.io/
- surajrmal 2y ago
  It's not always that simple. For a complicated self contained state machine this might work, but what about a situation where you are coordinating interactions with multiple distinct actors? You need a primitive to wait. At that point you need to pick a threading model.
  - 10000truths 2y ago
    State machines compose - the state of each actor is a part of your overall application state. So you'd create a giant state machine that contains a smaller state machine for each actor. Waits and timeouts can be handled by encapsulating time into your state. Then, you can feed your state machine with "X amount of time has elapsed" messages to inform it of the passage of time, and your state machine can emit something like a "sessions 123, 456 timed out" message as a response, and you can act on that by closing the sockets associated with those sessions.
surajrmal 2y ago
In languages where there is already considerable overhead I agree with you, but go style coroutines are not really a competitive option for some use cases. The alternative is callbacks or something else which is a lightweight abstraction on top of an event loop.
jfasydfw 2y ago
gevent was created over a decade ago to solve this. python users only have themselves to blame for ignoring it.
- westurner 2y ago
  From "Asyncio, twisted, tornado, gevent walk into a bar" (2023) https://news.ycombinator.com/item?id=37227567 https://news.ycombinator.com/item?id=37227567 : > IIRC the history of the async things in TLA in order: Twisted (callbacks), Eventlet (for Second Life by Linden Labs), tornado, gevent; gunicorn, Python 3.5+ asyncio, [uvicorn,] Async/await > History: https://en.wikipedia.org/wiki/Async/await https://en.wikipedia.org/wiki/Async/await

jonathan-adly 2y ago

We have traditionally used Django in all our projects. We believe it is one of the most underrated, beautifully designed, rock solid framework out there. However, if we are to be honest, the history of async usage in Django wasn't very impressive. You could argue that for most products, you don’t really need async. It was just an extra layer of complexity without any significant practical benefit. Over the last couple of years, AI use-cases have changed that perception. Many AI products have calling external APIs over the network as their bottleneck. This makes the complexity from async Python worth considering. FastAPI with its intuitive async usage and simplicity have risen to be the default API/web layer for AI projects. I wrote about using async Django in a relatively complex AI open source project here: https://jonathanadly.com/is-async-django-ready-for-prime-time https://jonathanadly.com/is-async-django-ready-for-prime-tim... tldr: Async django is ready! there is a couple of gotcha's here and there, but there should be no performance loss when using async Django instead of FastAPI for the same tasks. Django's built-in features greatly simplify and enhance the developer experience. So - go ahead and use async Django in your next project. It should be a lot smoother that it was a year or even six months ago.

fulafel 2y ago
(Disclaimer: I haven't used Django in a long time) Can you expand more on why these AI cases make the complexity tradeoff different? I'd imagine think waiting on a 3rd party LLM API call would be computationally very inexpensive compared to what's going on at the business end of that API call. Further lowering the cost, Django is usually configured to use multiple threads and/or processes so that this blocking call won't keep a CPU idle, no?
- flowingfocus 2y ago
  The explanation is in the article. Tldr is for sync functions, the CPU is blocked, with async functions, once the await statement is reached, other stuff can be handled in between
  - fulafel 2y ago
    Indeed it says "It enhances performance in areas where tasks are waiting for IO to complete by allowing the CPU to handle other tasks in the meantime". To restate my comment: I argued (1) this CPU cost would be very marginal compared to the LLM API compute cost, and (2) the CPU blocking claim doesn't really hold, due to the wonders of threads and processes.

andrewstuart 2y ago

Async Django is a bit of a puzzle …. who is it for? People who like synchronous Python can use Django. People who like asynch Python can use Starlette - the async web server also written by the guy who wrote Django. It’s not clear why Django needs to be async, especially when I get the sense there’s developers who like async and developers who prefer synch. It’s a natural fit for Django to fulfill the sync demand and Starlette to fulfill the async. They’re both good.

senko 2y ago
Starlette and Django are wildly different. Django has a batteries-includes approach, benefits from tighter integration of orm, auth, form handling, etc, and has a huge 3rd party ecosystem. First-class async support in Django allows Django users to avoid jumping through hoops (celery, channels, ...) for longer-running requests, something especially noticable if you're calling any kind of AI service.
- globular-toast 2y ago
  How much of that is relevant if you're going to be using django-ninja (pydantic) and the whole app has to be async, though? Django is fine for writing a thin CRUD layer around a database. It makes the easy stuff easy. But doesn't seem to help much for the hard stuff and often actively hinders it. Really the main reason for Django is its ORM and migrations. It's basically the other Python ORM (next to SQLAlchemy) but, unlike SQLAlchemy, it's not designed to be used standalone. In my experience I find Django (and active record ORMs in general) easier for people to get started with, but massively limiting in long run for complex domains.
  - senko 2y ago
    > if you're going to be using django-ninja (pydantic) This assumes that people don't do multi-page apps or sites any more, which ... isn't true. And I believe django-ninja replaces forms/serialization/deserialization and routing, while nicely integrating with everything else. > Django is fine for writing a thin CRUD layer around a database. In my dozen or so years with Django, I confess I did more than a few thin CRUD layers around a database. But also worked on billing systems for telecoms, insurance provider API services, live/on demand audio/video streaming services, a bunch of business process apps, AI codegen tools, and other web apps and API backends that were way more than thin CRUD layers around databases. Django was rarely a hindrance. In fact, Python being sync-only (or Django not supporting async) was usually more of a hindrance that anything Django specific. > In my experience I find Django (and active record ORMs in general) easier for people to get started with, but massively limiting in long run for complex domains. In my exprience the only situations where Django's ORM doesn't help much is when you have a lot of business logic in your database (views, stored procedures), or the database is organized in a way that's not Django's preffered way. Still works, mind you, just not as great a experience. However, the vast majority of projects I've encountered have none of those. Otherwise, I've found its ORM quite powerful, and easy to drop down to raw() in cases where you really need it (which was maybe 1% on the projects I've worked).

pacifika 2y ago

Learned quite a bit thanks!

olgeni 2y ago

I was using Daphne and then found out that it read the whole files in a POST before passing it along - needless to say I am no longer using Daphne \o/

beestripes 2y ago

Nearly all of the work done on Django ninja is by someone who lists their location on GitHub as Kharkiv. I hope that’s not current.