6 ms·

I feel this benchmark compares apples to oranges in some cases. For example, for node, the author puts a million promises into the runtime event loop and uses

I feel this benchmark compares apples to oranges in some cases. For example, for node, the author puts a million promises into the runtime event loop and uses `Promise.all` to wait for them all. This is very different from, say, the Go version where the author creates a million goroutines and puts `waitgroup.Done` as a defer call. While this might be the idiomatic way of concurrency in the respective languages, it does not account for how goroutines are fundamentally different from promises, and how the runtime does things differently. For JS, there's a single event loop. Counting the JS execution threads, the event loop thread and whatever else the runtime uses for async I/O, the execution model is fundamentally different from Go. Go (if not using `GOMAXPROCS`) spawns an OS thread for every physical thread that your machine has, and then uses a userspace scheduler to distribute goroutines to those threads. It may spawn more OS threads to account for OS threads sleeping on syscalls. Although I don't think the runtime will spawn extra threads in this case. It also depends on what the "concurrent tasks" (I know, concurrency != parallelism) are. Tasks such as reading a file or doing a network call are better done with something like promises, but CPU-bound tasks are better done with goroutines or Node worker_threads. It would be interesting to see how the memory usage changes when doing async I/O vs CPU-bound tasks concurrently in different languages.

SPascareli13 2y ago
As far as I know there is no way to do Promise like async in go, you HAVE to create a goroutine for each concurrent async task. If this is really the case then I believe the submition is valid. But I do think that spawning a goroutine just to do a non-blocking task and get its return is kinda wasteful.
- n2d4 2y ago
  You could in theory create your own event loop and then get the exact same behaviour as Promises in Go, but you probably shouldn't. Goroutines are the way to do this in Go, and it wouldn't be useful to benchmark code that would never be written in real life.
  - peterhon 2y ago
    I guess what you can do in golang that would be very similar to the rust impl would be this (and could be helpful even in real life, if all you need is a whole lot of timers): func test2(count int) { timers := make([]*time.Timer,count) for idx, _ := range timers { timers[idx] = time.NewTimer(10 * time.Second) } for idx, _ := range timers { <-timers[idx].C } } This yields to 263552 Maximum resident set size (kbytes) according to /usr/bin/time -v I'm not sure if I missed it, but I don't see the benchmark specify how the memory was measured, so I assumed the time -v.
threeseed 2y ago
The requirement is to run 1 million concurrent tasks. Of course each language will have a different way of achieving this task each of which will have their unique pros/cons. That's why we have these different languages to begin with.
- jakewins 2y ago
  The accounting here is weird though; Go isn’t using that RAM, it’s expecting the application to. The reason that doesn’t happen is because it’s a micro benchmark that produces no useful work.. The way the results are presented a reader may think the Go memory usage sounds equivalent to the others - boilerplate, ticket-to-play - and then the Go usage sounds super high. But they are not the same; that memory is in anticipation of a real world program using it
  - Aeolun 2y ago
    Isn’t that kind of dumb when none of the other languages do this? Apparently allocating memory is really fast? Maybe we should change the test to load 1MB of data in every task?
    - melodyogonna 2y ago
      Most of those languages (excepting Java virtual threads) uses stackless coroutines. Go uses stackful coroutines which allocates some memory upfront for a goroutine to use
      - gpderetta 2y ago
        Then it is fair to compare the memory usage of a stackful coroutine to a stack less one as they are the idiomatic way to perform async task on each language.
        jakewins 2y ago
        I mean this is subjective, but as long as it’s clear that one number is “this is the memory the runtime itself consumes to solve this problem” and the other number is “this is the runtime memory use and it includes pre-allocated stack space that a real application would then use”, sure Point being: Someone reading this to choose which runtime will fit their use case needs to be carefully to not assume the numbers measure the same thing. For some real world use cases the pre allocated stack will perform better than the runtimes that instead will do heap allocations.
n2d4 2y ago
Actually, I think this benchmark did the right thing, that I wish more benchmarks would do. I'm much less interested in what the differences between compilers are than in what the actual output will be if I ask a professional Go or Node.js dev to solve the same task. (TBF, it would've been better if the task benchmarked was something useful, eg. handling an HTTP request.) Go heavily encourages a certain kind of programming; JavaScript heavily encourages a different kind; and the article does a great job at showing what the consequences are.
- Quothling 2y ago
  > Go heavily encourages a certain kind of programming; True, but it really doesn't encourage you to run 1m goroutines with the standard memory setting. Though it's probably fair to run Go wastefully when you're comparing it to Promise.All.
  - n2d4 2y ago
    Of course! That's why the article is telling you that some languages (C#, Rust) are better at it than others (Go, Java). Doesn't mean that Go and Java are bad languages! Just that they aren't good to do this thing.
    - Quothling 2y ago
      The article is telling us that you can run really inefficient code. Goroutines should be run with worker pools and a buffered channel and it's silly to not do that and then compare it to things like an optimized Rust crate like Tokio.
      - Aeolun 2y ago
        Is that the ideomatic way to do it, or the best way you can imagine?
        Quothling 2y ago
gleenn 2y ago
Also, for Java, Virtual Threads are a very new feature (Java 21 IIRC or somewhere around there). OS threads have been around for decades. As a heavy JVM user it would have been nice to actually see those both broken out to compare as well!
- codetiger 2y ago
  The original benchmark had the comparison between Java thread and Java Virtual thread. https://pkolaczk.github.io/memory-consumption-of-async/ https://pkolaczk.github.io/memory-consumption-of-async/

SPascareli13 2y ago

As far as I know there is no way to do Promise like async in go, you HAVE to create a goroutine for each concurrent async task. If this is really the case then I believe the submition is valid. But I do think that spawning a goroutine just to do a non-blocking task and get its return is kinda wasteful.

n2d4 2y ago
You could in theory create your own event loop and then get the exact same behaviour as Promises in Go, but you probably shouldn't. Goroutines are the way to do this in Go, and it wouldn't be useful to benchmark code that would never be written in real life.
- peterhon 2y ago
  I guess what you can do in golang that would be very similar to the rust impl would be this (and could be helpful even in real life, if all you need is a whole lot of timers): func test2(count int) { timers := make([]*time.Timer,count) for idx, _ := range timers { timers[idx] = time.NewTimer(10 * time.Second) } for idx, _ := range timers { <-timers[idx].C } } This yields to 263552 Maximum resident set size (kbytes) according to /usr/bin/time -v I'm not sure if I missed it, but I don't see the benchmark specify how the memory was measured, so I assumed the time -v.

threeseed 2y ago

The requirement is to run 1 million concurrent tasks. Of course each language will have a different way of achieving this task each of which will have their unique pros/cons. That's why we have these different languages to begin with.

jakewins 2y ago
The accounting here is weird though; Go isn’t using that RAM, it’s expecting the application to. The reason that doesn’t happen is because it’s a micro benchmark that produces no useful work.. The way the results are presented a reader may think the Go memory usage sounds equivalent to the others - boilerplate, ticket-to-play - and then the Go usage sounds super high. But they are not the same; that memory is in anticipation of a real world program using it
- Aeolun 2y ago
  Isn’t that kind of dumb when none of the other languages do this? Apparently allocating memory is really fast? Maybe we should change the test to load 1MB of data in every task?
  - melodyogonna 2y ago
    Most of those languages (excepting Java virtual threads) uses stackless coroutines. Go uses stackful coroutines which allocates some memory upfront for a goroutine to use
    - gpderetta 2y ago
      Then it is fair to compare the memory usage of a stackful coroutine to a stack less one as they are the idiomatic way to perform async task on each language.
      - jakewins 2y ago
        I mean this is subjective, but as long as it’s clear that one number is “this is the memory the runtime itself consumes to solve this problem” and the other number is “this is the runtime memory use and it includes pre-allocated stack space that a real application would then use”, sure Point being: Someone reading this to choose which runtime will fit their use case needs to be carefully to not assume the numbers measure the same thing. For some real world use cases the pre allocated stack will perform better than the runtimes that instead will do heap allocations.

n2d4 2y ago

Actually, I think this benchmark did the right thing, that I wish more benchmarks would do. I'm much less interested in what the differences between compilers are than in what the actual output will be if I ask a professional Go or Node.js dev to solve the same task. (TBF, it would've been better if the task benchmarked was something useful, eg. handling an HTTP request.) Go heavily encourages a certain kind of programming; JavaScript heavily encourages a different kind; and the article does a great job at showing what the consequences are.

Quothling 2y ago
> Go heavily encourages a certain kind of programming; True, but it really doesn't encourage you to run 1m goroutines with the standard memory setting. Though it's probably fair to run Go wastefully when you're comparing it to Promise.All.
- n2d4 2y ago
  Of course! That's why the article is telling you that some languages (C#, Rust) are better at it than others (Go, Java). Doesn't mean that Go and Java are bad languages! Just that they aren't good to do this thing.
  - Quothling 2y ago
    The article is telling us that you can run really inefficient code. Goroutines should be run with worker pools and a buffered channel and it's silly to not do that and then compare it to things like an optimized Rust crate like Tokio.
    - Aeolun 2y ago
      Is that the ideomatic way to do it, or the best way you can imagine?
      - Quothling 2y ago

gleenn 2y ago

Also, for Java, Virtual Threads are a very new feature (Java 21 IIRC or somewhere around there). OS threads have been around for decades. As a heavy JVM user it would have been nice to actually see those both broken out to compare as well!

codetiger 2y ago
The original benchmark had the comparison between Java thread and Java Virtual thread. https://pkolaczk.github.io/memory-consumption-of-async/ https://pkolaczk.github.io/memory-consumption-of-async/