7 ms·

The problem I encounter with async/await (in JS) is that while an async method can call a non-async-function and do somewthing with the result of that, the reve

The problem I encounter with async/await (in JS) is that while an async method can call a non-async-function and do somewthing with the result of that, the reverse is not true, a sync function can call async-function but can not us the result of that in any way, except pass it on or upwards. What makes it worse is that you can not simply modify a sync-function to become an async-function, if it has existing callers because those would likely break them. This affects the whole tree of possible function calls in my program. If at some level I have a sync function but I see it needs to get to some data that only async fuction can provide, I may need to change a whole call-chain of my call-tree, not just modify a single function in that tree. Then as I develop my program I need to make the decision for every function; should it be sync or async? In many cases it could be either one so which should I choose? Making it async would seem to make it easier to evolve the program later. But then would it make sense to make every function async?

e1g 5d ago
Tactically, this problem is commonly known as “colored functions”[1], and the only option in JS is to have some other runtime coordinate your function execution; in JS, that solution is Effect[2] [1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... [2] https://effect.website/ https://effect.website/
- josephg 5d ago
  There are a lot of libraries which can help you deal with this, but ultimately the parent is right. I usually arrange my programs to have a call tree of all my async code, and separate call trees of sync processing work. You want to know ahead of time which is which. If you have a sync function which needs data that’s only available via a network request, take that data in as a function parameter or something. And make the caller responsible for making that data available before the function is called. It’s a simple model. It’s fast and quite easy to understand once you’re used to it. But you do need to plan ahead, and structure your programs with a plan.
  - RossBencina 5d ago
    I'm curious about your mental model. Would it be accurate to say that the async tree is the "IO program" and the sync functions operate on pure data, or is it more complicated than that?
    - josephg 4d ago
      Yeah, more or less. I think Haskell programmers are right on this.
  - e1g 5d ago
    A hallmark of good architecture is adaptability to unexpected changes in requirements. Planning ahead helps with 'known unknowns', but it's impractical when building across N years in a dynamic environment - "knowing ahead of time" is just not possible for anything non-trivial. You need strong architectural primitives that don't scale based on developers' omniscience. For example, say you have a workflow where, when someone signs up, you generate a user label, e.g., `$firstName $lastName`. You decide to move that to a function that might consider their personal title, preferred name, etc. Currently, it's a pure sync operation. Then, you discover people don't fill out that form at all, but some log in with Google, and you can use the name there as a fallback. Under flexible, this change is local: you can put that into your `createUserLabel(userInfo)`, and it can decide to fire off an API call to fill in any missing data, etc. In Promises land, this would taint the entire tree of everything everywhere that called that method, and all of those things must evolve or be refactored. In an Effectful system, this (previously unplanned) change remains isolated to that one function. Multiply that by every decision over multiple years for software looking for PMF, and requiring developers to "know ahead" severely slows down your ability to evolve.
    - simonask 4d ago
      Languages with effect systems typically let callers inherit the effects of their callees (e.g., calling an async function means the caller also is async), or force them to handle the effect (e.g., spawn the async call as a task and waiting synchronously for it to finish). Effects are just a generalization, where async/await is one particular effect. But: The fact that an operation now does some kind of I/O, or waits for user input, or whatever else you might express using async, has an _enormous_ impact on the architecture of your program. The “virality” of async is completely a feature, because it forces you to actually deal with that change, resulting in much more robust software. It’s “inconvenient” because the architecture of your program changed. That’s what the job is, though. Languages that don’t help you here (by hiding that you made a change with huge ramifications) make it actively harder to deliver working software, in my opinion. You get there faster, but it won’t keep working.
brabel 4d ago
This is often brought up as if it were a problem, but I see async functions as something similar to IO in Haskell. Almost all asynchronous functions I write are asynchronous because they will do IO of some sort. Async functions end up being markers of where IO may occur, which is very useful. It is very rare that I need to change a function from being sync to async (and the inverse pretty much never happens), and when that happens it's usually not a big deal (the caller is highly likely to be an async function within a short stack distance, so only one or two functions in the middle normally need to change). In summary, async is something that looks problematic in theory, but in practice it just works really well!
- mrsmrtss 4d ago
  Agreed on async. You better know if a function does IO, hiding that can lead to nasty surprises.
- jeremyjh 4d ago
  This is only true in Javascript though - even though you have the same function coloring aspect in most other languages with async/await, the other ones do not come with this benefit since synchronous I/O is not only possible but the classical default.
  - brabel 4d ago
    I mostly do this in Dart (though even Dart also has sync IO, it’s just not supposed to be used often), but yeah other languages may not have this benefit.
  - hombre_fatal 4d ago
    Javascript's async-everything is really unique in a domain where async is almost always bolted on to synchronous-everything in some sort of incompatible subecosystem.
  - skybrian 3d ago
    Also, it’s only true in the browser. Node and Deno have sync I/O. You still end up having to make a choice about whether your function should be async.
whilenot-dev 4d ago
> But then would it make sense to make every function async? No, that doesn't make sense at all! You're being too reductionist... I/O has its place in every real world program, and the true limitation (or what you call a "problem") are the single-threaded runtimes of JavaScript. It's not a question of whether you should mark your functions async or not, as if it's an issue of consistency in the call-tree of your program. The true question should rather be whether your functions are I/O-bound (and would actually block the single-threaded event loop) or are solely compute-bound. You're forgetting the fact that async/await in JavaScript was a historical design choice to prevent the callback-hell that came with single-threaded concurrency. So if you'd want to get back to that callback-hell (and convert async functions back to "some-form-of" sync), you still can[0]: // some dummy async function that doesn't really do any I/O async function add( a: number, b: number, ): Promise<number> { return a + b; } // convert async function back to sync to enjoy callback-hell again function addUnpromisified( a: number, b: number, cb: ((result: number | null, reason: any) => any), ): void { add(a, b) .then((result) => { cb(result, null); }) .catch((reason) => { cb(null, reason); }); } You can think of async/await as an evolution of generators[1], as every await yields control back to the event loop. I'd actually encourage you to write some of your programs' behavior with generators if you've never done that before. Generator functions will yield control back to the caller, which is an interesting way to design programs when the caller is you instead of the event loop. It's in Python, but David Beazley still has one of the best explanations on that topic, and shows the how and why you'd want to design an event loop live on stage[2]. [0]: https://www.typescriptlang.org/play/?#code/PTAEGcHsFsFNQCYFdrQJ6gIbjQOwMagBmSBALgJaS6hkAWmZiks4uA5EwE6yYA2fDAkhZcGAJLAA8gChseQiXJUamBAgAUM0FgBcoXCgBGsLgBptoI-sPQT5mQEp9ABS4wK4WAB5b9gHygAN6WPGRIXKqgANRWANwyAL4yMiCg+NQAbqZM8gTEpPiU1FaY+ADWtCI4+WQisLgAVpAY+Px8RmXlALR0sAJYAOaYFLgySkUqWOoAqrgADu7QnhREFLCalpg2xqYWOtYGuw46+IcaGjzgSHxkO3amoAA+RwJmoDzY1PqYYo6gAF5Ar80I4LM5QJlIBQEMEtuoNJh3kZHJYdAA6egNC5XG5kf5A4LpIyXVh496GASOOKgRKonQYtpkfB0HG8KC4AmBILEjSUvjvT4c6m06lJIA https://www.typescriptlang.org/play/?#code/PTAEGcHsFsFNQCYFd... [1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_generators#generator_functions https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid... [2]: https://www.youtube.com/watch?v=MCs5OvhV9S4 https://www.youtube.com/watch?v=MCs5OvhV9S4
- cypherpunk666 3d ago
jayd16 4d ago
Async/await syntax doesn't guarantee what kind mechanism is servicing the async work. Consider a situation where you have limited threads (perhaps even only one servicing both sync and async calls). Blocking that thread to wait for an async call would prevent the async call from completing and would be a deadlock. As noted, you can run sync code from an async context. Even if you have a thread pool, blocking in sync has a chance to block and consume an async worker thread. That can also cause deadlocks. They don't make it easy to wait synchronously because it's a bad idea. Making the syntax more amicable to blocking is a worse idea.

e1g 5d ago

Tactically, this problem is commonly known as “colored functions”[1], and the only option in JS is to have some other runtime coordinate your function execution; in JS, that solution is Effect[2] [1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/ https://journal.stuffwithstuff.com/2015/02/01/what-color-is-... [2] https://effect.website/ https://effect.website/

josephg 5d ago
There are a lot of libraries which can help you deal with this, but ultimately the parent is right. I usually arrange my programs to have a call tree of all my async code, and separate call trees of sync processing work. You want to know ahead of time which is which. If you have a sync function which needs data that’s only available via a network request, take that data in as a function parameter or something. And make the caller responsible for making that data available before the function is called. It’s a simple model. It’s fast and quite easy to understand once you’re used to it. But you do need to plan ahead, and structure your programs with a plan.
- RossBencina 5d ago
  I'm curious about your mental model. Would it be accurate to say that the async tree is the "IO program" and the sync functions operate on pure data, or is it more complicated than that?
  - josephg 4d ago
    Yeah, more or less. I think Haskell programmers are right on this.
- e1g 5d ago
  A hallmark of good architecture is adaptability to unexpected changes in requirements. Planning ahead helps with 'known unknowns', but it's impractical when building across N years in a dynamic environment - "knowing ahead of time" is just not possible for anything non-trivial. You need strong architectural primitives that don't scale based on developers' omniscience. For example, say you have a workflow where, when someone signs up, you generate a user label, e.g., `$firstName $lastName`. You decide to move that to a function that might consider their personal title, preferred name, etc. Currently, it's a pure sync operation. Then, you discover people don't fill out that form at all, but some log in with Google, and you can use the name there as a fallback. Under flexible, this change is local: you can put that into your `createUserLabel(userInfo)`, and it can decide to fire off an API call to fill in any missing data, etc. In Promises land, this would taint the entire tree of everything everywhere that called that method, and all of those things must evolve or be refactored. In an Effectful system, this (previously unplanned) change remains isolated to that one function. Multiply that by every decision over multiple years for software looking for PMF, and requiring developers to "know ahead" severely slows down your ability to evolve.
  - simonask 4d ago
    Languages with effect systems typically let callers inherit the effects of their callees (e.g., calling an async function means the caller also is async), or force them to handle the effect (e.g., spawn the async call as a task and waiting synchronously for it to finish). Effects are just a generalization, where async/await is one particular effect. But: The fact that an operation now does some kind of I/O, or waits for user input, or whatever else you might express using async, has an _enormous_ impact on the architecture of your program. The “virality” of async is completely a feature, because it forces you to actually deal with that change, resulting in much more robust software. It’s “inconvenient” because the architecture of your program changed. That’s what the job is, though. Languages that don’t help you here (by hiding that you made a change with huge ramifications) make it actively harder to deliver working software, in my opinion. You get there faster, but it won’t keep working.

brabel 4d ago

This is often brought up as if it were a problem, but I see async functions as something similar to IO in Haskell. Almost all asynchronous functions I write are asynchronous because they will do IO of some sort. Async functions end up being markers of where IO may occur, which is very useful. It is very rare that I need to change a function from being sync to async (and the inverse pretty much never happens), and when that happens it's usually not a big deal (the caller is highly likely to be an async function within a short stack distance, so only one or two functions in the middle normally need to change). In summary, async is something that looks problematic in theory, but in practice it just works really well!

mrsmrtss 4d ago
Agreed on async. You better know if a function does IO, hiding that can lead to nasty surprises.
jeremyjh 4d ago
This is only true in Javascript though - even though you have the same function coloring aspect in most other languages with async/await, the other ones do not come with this benefit since synchronous I/O is not only possible but the classical default.
- brabel 4d ago
  I mostly do this in Dart (though even Dart also has sync IO, it’s just not supposed to be used often), but yeah other languages may not have this benefit.
- hombre_fatal 4d ago
  Javascript's async-everything is really unique in a domain where async is almost always bolted on to synchronous-everything in some sort of incompatible subecosystem.
- skybrian 3d ago
  Also, it’s only true in the browser. Node and Deno have sync I/O. You still end up having to make a choice about whether your function should be async.

whilenot-dev 4d ago

> But then would it make sense to make every function async? No, that doesn't make sense at all! You're being too reductionist... I/O has its place in every real world program, and the true limitation (or what you call a "problem") are the single-threaded runtimes of JavaScript. It's not a question of whether you should mark your functions async or not, as if it's an issue of consistency in the call-tree of your program. The true question should rather be whether your functions are I/O-bound (and would actually block the single-threaded event loop) or are solely compute-bound. You're forgetting the fact that async/await in JavaScript was a historical design choice to prevent the callback-hell that came with single-threaded concurrency. So if you'd want to get back to that callback-hell (and convert async functions back to "some-form-of" sync), you still can[0]: // some dummy async function that doesn't really do any I/O async function add( a: number, b: number, ): Promise<number> { return a + b; } // convert async function back to sync to enjoy callback-hell again function addUnpromisified( a: number, b: number, cb: ((result: number | null, reason: any) => any), ): void { add(a, b) .then((result) => { cb(result, null); }) .catch((reason) => { cb(null, reason); }); } You can think of async/await as an evolution of generators[1], as every await yields control back to the event loop. I'd actually encourage you to write some of your programs' behavior with generators if you've never done that before. Generator functions will yield control back to the caller, which is an interesting way to design programs when the caller is you instead of the event loop. It's in Python, but David Beazley still has one of the best explanations on that topic, and shows the how and why you'd want to design an event loop live on stage[2]. [0]: https://www.typescriptlang.org/play/?#code/PTAEGcHsFsFNQCYFdrQJ6gIbjQOwMagBmSBALgJaS6hkAWmZiks4uA5EwE6yYA2fDAkhZcGAJLAA8gChseQiXJUamBAgAUM0FgBcoXCgBGsLgBptoI-sPQT5mQEp9ABS4wK4WAB5b9gHygAN6WPGRIXKqgANRWANwyAL4yMiCg+NQAbqZM8gTEpPiU1FaY+ADWtCI4+WQisLgAVpAY+Px8RmXlALR0sAJYAOaYFLgySkUqWOoAqrgADu7QnhREFLCalpg2xqYWOtYGuw46+IcaGjzgSHxkO3amoAA+RwJmoDzY1PqYYo6gAF5Ar80I4LM5QJlIBQEMEtuoNJh3kZHJYdAA6egNC5XG5kf5A4LpIyXVh496GASOOKgRKonQYtpkfB0HG8KC4AmBILEjSUvjvT4c6m06lJIA https://www.typescriptlang.org/play/?#code/PTAEGcHsFsFNQCYFd... [1]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Iterators_and_generators#generator_functions https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid... [2]: https://www.youtube.com/watch?v=MCs5OvhV9S4 https://www.youtube.com/watch?v=MCs5OvhV9S4

cypherpunk666 3d ago

jayd16 4d ago

Async/await syntax doesn't guarantee what kind mechanism is servicing the async work. Consider a situation where you have limited threads (perhaps even only one servicing both sync and async calls). Blocking that thread to wait for an async call would prevent the async call from completing and would be a deadlock. As noted, you can run sync code from an async context. Even if you have a thread pool, blocking in sync has a chance to block and consume an async worker thread. That can also cause deadlocks. They don't make it easy to wait synchronously because it's a bad idea. Making the syntax more amicable to blocking is a worse idea.