6 ms·

Coroutines and Fibers: Why and When

57 points by stelabouras 11y ago

iambvk 11y ago
I don't understand how resumable functions can be used to build stack-full coroutines, like in Go language, can anyone give an example or some links?
- gepoch 11y ago
  Go has an interesting history surrounding its stacks. Originally the stack was a doubly linked-list (segmented stack), but it changed into a more vector-like format (contiguous stacks) in 1.3. Here's a pretty good write-up: http://agis.io/2014/03/25/contiguous-stacks-in-go.html http://agis.io/2014/03/25/contiguous-stacks-in-go.html
vvanders 11y ago
Coroutines are also fantastic for scripting Game AI: local player = nil while(player == nil) do yeild(0) player = LookForPlayer(...) end while(WalkTowardsPlayer(player)) do yeild(0) end while(!PlayerDead(player)) do MeleePlayer(player) yield(0) end Makes state management super-simple and is something you can easily teach to designers as well. This is partly why you see Lua show up in so many game engines.
- striking 11y ago
  FSM may be a better fit for this specific application, though.
  - vvanders 11y ago
    For something engineered, sure. However if you're working with designers who don't have a lot of formal education it's much easier to convey the concept of a function that "pauses" where yield() happens. Also quite a bit of gameplay code ends up as throw-away so being able to quickly put behaviors together is a plus.
    - striking 11y ago
      You're saying a tree of FSM states is more difficult to convey than linearized coroutine code, with emphasis on informal programmers/designers? You may want to take a look at visual FSM solutions, which do what you specify and more, easily. You're not even wrong. https://unity3d.com/learn/tutorials/modules/beginner/animation/animator-scripting https://unity3d.com/learn/tutorials/modules/beginner/animati..., for an example in an Animation program
dblohm7 11y ago
One caveat with coroutines and fibers is that all the code that runs on them must be fully aware that they are running on said coroutines and fibers.
- willtim 11y ago
  Not true if one is using Haskell (or any other sufficiently expressive language)
  - pron 11y ago
    That has nothing to do with Haskell's expressivity and everything to do with its full purity. If you look at, say, OCaml, the code (or at least the libraries) must also be aware that it's running in a fiber, just as in Java, Go or Erlang. While full purity has its virtues, it also has serious drawbacks, and it's unclear at this point whether or not at the end of the day purity is a net-gain, neutral, or even a net-loss.
- nqzero 11y ago
  at least in java (i'm using https://github.com/kilim/kilim https://github.com/kilim/kilim, but quasar appears to be similar), only methods that yield need to be aware
  - vbit 11y ago
    Same for Lua.
- qewrffewqwfqew 11y ago
  Under what language / toolkit? Folks are pointing out counterexamples below and I could name a few myself, but I'm curious where you're coming from :-)
vbit 11y ago
I generally prefer coroutines myself but this article doesn't cover some points well. 'Fair scheduling' - nobody writes single threaded programs that process requests sequentially. It's either a thread-per-request system vs. a coroutine-per-request system. Threads are generally going to be more fair at scheduling since they are preemptive. Most coroutines are cooperative (Python, Lua) - so a long running coroutine (stuck doing some CPU cycle) will block all other inflight requests and cause latency variance. Some systems are preemptive (Erlang) so they don't suffer from the variance. The benefit of coroutines is you can have a very large number of coroutines, probably orders of magnitude more than the number of threads - so coroutine-per-request models will scale much more than threads-per-request. The article is spot on about the context switching cost - it's so much cheaper to switch between coroutines. You can also use multiple request per threads and use async IO, but that's the same as using coroutines with a much worse programming model.
- avdicius 11y ago
  Yes, fibers/coroutines are good for socket I/O because sockets could be made non-blocking, therefore a fiber could yield on EWOULDBLOCK error from a socket. For file I/O there is no non-blocking I/O option, so the whole OS-thread might block on it along with all the fibers that it owns. Therefore any operation that is not socket I/O and goes beyond CPU and RAM should be delegated from fibers to a good old worker thread pool. The fiber should yield after submitting a request to the pool, and on the request completion the worker thread should notify the fiber scheduler to resume the original fiber.
  - pas 11y ago
    Well, there's no POSIX async file I/O, but there's libaio for Linux's aio syscalls (io_submit and co.), which works. (MySQL/InnoDB uses it for example.)
    - avdicius 11y ago
      Upon return from a non-blocking I/O call the request is either failed or succeeded. Upon return from an async I/O call the request might additionally be in progress thus the fiber is required to wait for its completion. This case has little practical difference from a thread pool. Except with a thread pool it is the worker thread that notifies the fiber scheduler on I/O completion. With async I/O it is the kernel that notifies the application and then the application (from a signal handler or from an auxiliary thread) should propagate the notification to the fiber scheduler.
deleted 11y ago
[deleted]
nqzero 11y ago
i'm using java fibers (https://github.com/kilim/kilim https://github.com/kilim/kilim) for a database for anyone interested in using fibers inside a java webapp, i just wrote up a quick survey of async java servlet performance: http://blog.nqzero.com/2016/01/asynchronous-java-webserver-hello-world.html http://blog.nqzero.com/2016/01/asynchronous-java-webserver-h... (based TechEmpower plaintext benchmark, but limited to java async) the model is that you use async in the server, and then bridge to a fiber implementation and then complete the async response when the fiber completes. for handling high latency low cpu processing on the server, this technique allows handling a huge number of connections
- namelezz 11y ago
  What do you think of Quasar fiber[1]? [1] http://docs.paralleluniverse.co/quasar/#fibers http://docs.paralleluniverse.co/quasar/#fibers
  - nqzero 11y ago
    i assume that it's comparable to kilim, but have limited experience Comsat (one of the servers in my benchmark i linked) is written by the same people and uses Quasar. so eg Comsat Jetty is an async jetty servlet with a bridge to a Quasar fiber-based handler. the performance in my simplistic test wasn't great (async jetty was 50% faster than comsat jetty) - i don't know if that's representative however, it's very easy to work with and mimics the Jetty API, which is really nice. https://github.com/nqzero/jempower/blob/master/comsat/src/main/java/ComsatJetty.java https://github.com/nqzero/jempower/blob/master/comsat/src/ma... for my database i was initially worried about the LGPL license for Quasar, but the guys assured me that it's not viral in this usage and i think they're correct pron (one of the developers) is active posting on HN and has written some very good/knowledgeable stuff about java, so i suspect that Quasar is technically solid. i do plan on doing an integration with my database at some point and i'll write up my results and post them on HN
avdicius 11y ago
Coroutines seem to be a very popular topic. And again my shameless plug, I did exactly the thing described in this article in my project: https://github.com/ademakov/MainMemory https://github.com/ademakov/MainMemory
TheGrassyKnoll 11y ago
For Python: (might help someone) http://www.dabeaz.com/coroutines/ http://www.dabeaz.com/coroutines/

57 points by stelabouras 11y ago

I don't understand how resumable functions can be used to build stack-full coroutines, like in Go language, can anyone give an example or some links?

gepoch 11y ago
Go has an interesting history surrounding its stacks. Originally the stack was a doubly linked-list (segmented stack), but it changed into a more vector-like format (contiguous stacks) in 1.3. Here's a pretty good write-up: http://agis.io/2014/03/25/contiguous-stacks-in-go.html http://agis.io/2014/03/25/contiguous-stacks-in-go.html

vvanders 11y ago

Coroutines are also fantastic for scripting Game AI: local player = nil while(player == nil) do yeild(0) player = LookForPlayer(...) end while(WalkTowardsPlayer(player)) do yeild(0) end while(!PlayerDead(player)) do MeleePlayer(player) yield(0) end Makes state management super-simple and is something you can easily teach to designers as well. This is partly why you see Lua show up in so many game engines.

striking 11y ago
FSM may be a better fit for this specific application, though.
- vvanders 11y ago
  For something engineered, sure. However if you're working with designers who don't have a lot of formal education it's much easier to convey the concept of a function that "pauses" where yield() happens. Also quite a bit of gameplay code ends up as throw-away so being able to quickly put behaviors together is a plus.
  - striking 11y ago
    You're saying a tree of FSM states is more difficult to convey than linearized coroutine code, with emphasis on informal programmers/designers? You may want to take a look at visual FSM solutions, which do what you specify and more, easily. You're not even wrong. https://unity3d.com/learn/tutorials/modules/beginner/animation/animator-scripting https://unity3d.com/learn/tutorials/modules/beginner/animati..., for an example in an Animation program

dblohm7 11y ago

One caveat with coroutines and fibers is that all the code that runs on them must be fully aware that they are running on said coroutines and fibers.

willtim 11y ago
Not true if one is using Haskell (or any other sufficiently expressive language)
- pron 11y ago
  That has nothing to do with Haskell's expressivity and everything to do with its full purity. If you look at, say, OCaml, the code (or at least the libraries) must also be aware that it's running in a fiber, just as in Java, Go or Erlang. While full purity has its virtues, it also has serious drawbacks, and it's unclear at this point whether or not at the end of the day purity is a net-gain, neutral, or even a net-loss.
nqzero 11y ago
at least in java (i'm using https://github.com/kilim/kilim https://github.com/kilim/kilim, but quasar appears to be similar), only methods that yield need to be aware
- vbit 11y ago
  Same for Lua.
qewrffewqwfqew 11y ago
Under what language / toolkit? Folks are pointing out counterexamples below and I could name a few myself, but I'm curious where you're coming from :-)

vbit 11y ago

I generally prefer coroutines myself but this article doesn't cover some points well. 'Fair scheduling' - nobody writes single threaded programs that process requests sequentially. It's either a thread-per-request system vs. a coroutine-per-request system. Threads are generally going to be more fair at scheduling since they are preemptive. Most coroutines are cooperative (Python, Lua) - so a long running coroutine (stuck doing some CPU cycle) will block all other inflight requests and cause latency variance. Some systems are preemptive (Erlang) so they don't suffer from the variance. The benefit of coroutines is you can have a very large number of coroutines, probably orders of magnitude more than the number of threads - so coroutine-per-request models will scale much more than threads-per-request. The article is spot on about the context switching cost - it's so much cheaper to switch between coroutines. You can also use multiple request per threads and use async IO, but that's the same as using coroutines with a much worse programming model.

avdicius 11y ago
Yes, fibers/coroutines are good for socket I/O because sockets could be made non-blocking, therefore a fiber could yield on EWOULDBLOCK error from a socket. For file I/O there is no non-blocking I/O option, so the whole OS-thread might block on it along with all the fibers that it owns. Therefore any operation that is not socket I/O and goes beyond CPU and RAM should be delegated from fibers to a good old worker thread pool. The fiber should yield after submitting a request to the pool, and on the request completion the worker thread should notify the fiber scheduler to resume the original fiber.
- pas 11y ago
  Well, there's no POSIX async file I/O, but there's libaio for Linux's aio syscalls (io_submit and co.), which works. (MySQL/InnoDB uses it for example.)
  - avdicius 11y ago
    Upon return from a non-blocking I/O call the request is either failed or succeeded. Upon return from an async I/O call the request might additionally be in progress thus the fiber is required to wait for its completion. This case has little practical difference from a thread pool. Except with a thread pool it is the worker thread that notifies the fiber scheduler on I/O completion. With async I/O it is the kernel that notifies the application and then the application (from a signal handler or from an auxiliary thread) should propagate the notification to the fiber scheduler.

deleted 11y ago

[deleted]

nqzero 11y ago

i'm using java fibers (https://github.com/kilim/kilim https://github.com/kilim/kilim) for a database for anyone interested in using fibers inside a java webapp, i just wrote up a quick survey of async java servlet performance: http://blog.nqzero.com/2016/01/asynchronous-java-webserver-hello-world.html http://blog.nqzero.com/2016/01/asynchronous-java-webserver-h... (based TechEmpower plaintext benchmark, but limited to java async) the model is that you use async in the server, and then bridge to a fiber implementation and then complete the async response when the fiber completes. for handling high latency low cpu processing on the server, this technique allows handling a huge number of connections

namelezz 11y ago
What do you think of Quasar fiber[1]? [1] http://docs.paralleluniverse.co/quasar/#fibers http://docs.paralleluniverse.co/quasar/#fibers
- nqzero 11y ago
  i assume that it's comparable to kilim, but have limited experience Comsat (one of the servers in my benchmark i linked) is written by the same people and uses Quasar. so eg Comsat Jetty is an async jetty servlet with a bridge to a Quasar fiber-based handler. the performance in my simplistic test wasn't great (async jetty was 50% faster than comsat jetty) - i don't know if that's representative however, it's very easy to work with and mimics the Jetty API, which is really nice. https://github.com/nqzero/jempower/blob/master/comsat/src/main/java/ComsatJetty.java https://github.com/nqzero/jempower/blob/master/comsat/src/ma... for my database i was initially worried about the LGPL license for Quasar, but the guys assured me that it's not viral in this usage and i think they're correct pron (one of the developers) is active posting on HN and has written some very good/knowledgeable stuff about java, so i suspect that Quasar is technically solid. i do plan on doing an integration with my database at some point and i'll write up my results and post them on HN

avdicius 11y ago

Coroutines seem to be a very popular topic. And again my shameless plug, I did exactly the thing described in this article in my project: https://github.com/ademakov/MainMemory https://github.com/ademakov/MainMemory

TheGrassyKnoll 11y ago

For Python: (might help someone) http://www.dabeaz.com/coroutines/ http://www.dabeaz.com/coroutines/