5 ms·

we are using libco[1] in our project[2]; it provides a good abstraction for architectures and provide different backends using ucontext, setjmp and native regis

we are using libco[1] in our project[2]; it provides a good abstraction for architectures and provide different backends using ucontext, setjmp and native register management for x86 and x86_64. [1] https://byuu.org/library/libco/ https://byuu.org/library/libco/ [2] https://fluentbit.io https://fluentbit.io

ioquatix 8y ago
I like `libco`. The API is a bit more heavy weight than what I submitted to Ruby. That's because `libco` tries to avoid an assembler and instead dynamically loads binary data into an executable section. In my opinion it's an unnecessary performance cost. The benefit is to avoid an assembler at compile time. I think it's better to use natively compiled code by the normal toolchain. I think it's a simpler design.
- byuu 8y ago
  It's a very tough tradeoff. You're not the first to take aim at it, and that's fair. Visual C++, GCC/Clang, and Intel C++ have quite different syntax for declaring inline assembler. I did not wish to burden libco users with having to use a specific compiler. Further, GCC/Clang can prove ... challenging to produce "naked" functions (without stack BP/SP adjustments), which is critical for libco's co_switch routine. I recall having trouble with a less popular OS. Using an external assembler like yasm or GNU as adds further dependencies on specific tools. It is a very minor performance penalty to have the co_swap thunk, but it is non-zero, so I respect your decision. But do note at least that libco supports many architectures (without going pointlessly obscure.) It'd be a shame to miss out on cooperating in supporting architectures by duplicating our work over something like this. With or without this, coroutines will always be slaughtered compared to a simple stackless state machine function, but they really shine and (in my opinion) are worth their overhead when you need to switch tasks in the middle of nested function calls. ... # Put the first argument into the return value movq %rdi, %rax # We pop the return address and jump to it ret So if you look at libco: https://gitlab.com/higan/higan/blob/master/libco/amd64.c#L26 https://gitlab.com/higan/higan/blob/master/libco/amd64.c#L26 You'll notice that I get the return address into RAX as soon as possible. Believe it or not, this makes a real difference in performance. It allows the CPU to start fetching instructions after the JMP/RET even sooner than if you have it at the bottom of the function as you do. The choice between push/pop and mov [rsp] for handling the non-volatile registers isn't really important. I found the latter slightly more performant on Athlon 64 CPUs, and a wash on Intel CPUs. Preserving signals isn't so critical, but SSE is probably quite important, and only necessary on Windows. Again, your decision, but I would preserve it in your case. Can lead to really nasty surprises if you don't.
  - gpderetta 8y ago
    >You'll notice that I get the return address into RAX as soon as possible. Believe it or not, this makes a real difference in performance. It allows the CPU to start fetching instructions after the JMP/RET even sooner than if you have it at the bottom of the function as you do. The "execution" of the jump happens well before the pop rax is executed. The real difference is using a jump insted of a ret. The latter will be always mispredicted as the CPU return address predictor (the stack engine) will always get it wrong, while the indirect predictor used by jmp has a chance to get it right.
    - ioquatix 8y ago
      Do you both mind elaborating a bit on this? I have to admit CPU instruction scheduling is not really my area.. I'd love to hear more.
      - byuu 8y ago
        It's been about ten years since I made that code change, so I'm a bit fuzzy on the details, but in my mind, I really don't see how "pop rax; jmp rax" is going to be faster than "ret". I really feel like it was important to get the return address into rax as soon as possible. And that, plus the ability to save/restore the xmm registers for the Win64 ABI (there is no push/pop xmm#), is why I used mov/movaps[] instead. I do recall it also testing faster on Athlons as well, but that was so long ago as to be irrelevant now, most likely. Nostalgic! :3 In any case, I'd recommend benchmarking both under an application that tries to do as many cothread switches as possible per second. Theory's one thing, seeing the results in practice is always another.
        ioquatix 8y ago
        If you take a look, you'll see that I specifically designed the API to avoid the need for any global/thread local state. So, the argument to `coroutine_transfer` is passed to the coroutine and the calling coroutine is returned from `coroutine_transfer`. So, using %rax is not possible, because it's the return value. That being said, another register would be fine. So you think there is a performance improvement from putting the return address specifically into %rax as soon as possible?

ioquatix 8y ago

I like `libco`. The API is a bit more heavy weight than what I submitted to Ruby. That's because `libco` tries to avoid an assembler and instead dynamically loads binary data into an executable section. In my opinion it's an unnecessary performance cost. The benefit is to avoid an assembler at compile time. I think it's better to use natively compiled code by the normal toolchain. I think it's a simpler design.

byuu 8y ago
It's a very tough tradeoff. You're not the first to take aim at it, and that's fair. Visual C++, GCC/Clang, and Intel C++ have quite different syntax for declaring inline assembler. I did not wish to burden libco users with having to use a specific compiler. Further, GCC/Clang can prove ... challenging to produce "naked" functions (without stack BP/SP adjustments), which is critical for libco's co_switch routine. I recall having trouble with a less popular OS. Using an external assembler like yasm or GNU as adds further dependencies on specific tools. It is a very minor performance penalty to have the co_swap thunk, but it is non-zero, so I respect your decision. But do note at least that libco supports many architectures (without going pointlessly obscure.) It'd be a shame to miss out on cooperating in supporting architectures by duplicating our work over something like this. With or without this, coroutines will always be slaughtered compared to a simple stackless state machine function, but they really shine and (in my opinion) are worth their overhead when you need to switch tasks in the middle of nested function calls. ... # Put the first argument into the return value movq %rdi, %rax # We pop the return address and jump to it ret So if you look at libco: https://gitlab.com/higan/higan/blob/master/libco/amd64.c#L26 https://gitlab.com/higan/higan/blob/master/libco/amd64.c#L26 You'll notice that I get the return address into RAX as soon as possible. Believe it or not, this makes a real difference in performance. It allows the CPU to start fetching instructions after the JMP/RET even sooner than if you have it at the bottom of the function as you do. The choice between push/pop and mov [rsp] for handling the non-volatile registers isn't really important. I found the latter slightly more performant on Athlon 64 CPUs, and a wash on Intel CPUs. Preserving signals isn't so critical, but SSE is probably quite important, and only necessary on Windows. Again, your decision, but I would preserve it in your case. Can lead to really nasty surprises if you don't.
- gpderetta 8y ago
  >You'll notice that I get the return address into RAX as soon as possible. Believe it or not, this makes a real difference in performance. It allows the CPU to start fetching instructions after the JMP/RET even sooner than if you have it at the bottom of the function as you do. The "execution" of the jump happens well before the pop rax is executed. The real difference is using a jump insted of a ret. The latter will be always mispredicted as the CPU return address predictor (the stack engine) will always get it wrong, while the indirect predictor used by jmp has a chance to get it right.
  - ioquatix 8y ago
    Do you both mind elaborating a bit on this? I have to admit CPU instruction scheduling is not really my area.. I'd love to hear more.
    - byuu 8y ago
      It's been about ten years since I made that code change, so I'm a bit fuzzy on the details, but in my mind, I really don't see how "pop rax; jmp rax" is going to be faster than "ret". I really feel like it was important to get the return address into rax as soon as possible. And that, plus the ability to save/restore the xmm registers for the Win64 ABI (there is no push/pop xmm#), is why I used mov/movaps[] instead. I do recall it also testing faster on Athlons as well, but that was so long ago as to be irrelevant now, most likely. Nostalgic! :3 In any case, I'd recommend benchmarking both under an application that tries to do as many cothread switches as possible per second. Theory's one thing, seeing the results in practice is always another.
      - ioquatix 8y ago
        If you take a look, you'll see that I specifically designed the API to avoid the need for any global/thread local state. So, the argument to `coroutine_transfer` is passed to the coroutine and the calling coroutine is returned from `coroutine_transfer`. So, using %rax is not possible, because it's the return value. That being said, another register would be fine. So you think there is a performance improvement from putting the return address specifically into %rax as soon as possible?