8 ms·

lol please no If you are using C++, and want to parallelize something, just add "std::execution::par" to your algorithms. Instead of writing "std::for_each(..

lol please no If you are using C++, and want to parallelize something, just add "std::execution::par" to your algorithms. Instead of writing "std::for_each(...)" just write "std::for_each(std::execution::par, ...)". That's it. It really is that simple. And with the right compilers you can just compile the code to run on FPGAS, GPUs, or whatever. For someone that knows C++, doing that is the lowest barrier of entry, and gets you most of the way there without having to learn "some other programming language" like OpenMP (or anything else).

dragontamer 4y ago
OpenMP can parellize a for loop like: #pragma openmp parallel for for(int i=0; i<1000000; i++) C[i] = A[i] + B[i]; It's normal C++ and the pragma auto-parallelizes the loop. It's actually really easy and convenient. OpenMP is probably the easiest join/fork model of parallelism on any C++ system I've used. It doesn't always get the best utilization of your CPU cores, but it's really, really simple. It's the best way to start IMO, far easier than std::thread, or switching to functional style for other libraries. Just write the same code as before but with a few #pragma omp statements here and there.
- maxwell86 4y ago
  > It's normal C++ C++ is an ISO standard, you can use it _everywhere_, in space, in automotive, in aviation, in trains, in medical devices, _everywhere_. OpenMP is not C++, it is a different programming language than C++. OpenMP is not an ISO standard, you can't use it in _most_ domains that you can use C++. Your example: #pragma openmp parallel for for(int i=0; i<1000000; i++) C[i] = A[i] + B[i]; shows how bad OpenMP is. It does not run in parallel on GPUs or on FPGAS (lacking target offload directives), and you can't use it on most domains in which you can use C++. The following is ISO standard C++: std::for_each_n(std::execution::par, std::ranges::iota(0).begin(), 1000000, [](int i) { C[i] = A[i] + B[i]; }); it runs in _parallel_ EVERYWHERE: GPUs, CPUs, FPGAS, and it is certified for all domains for which C++ is (that is: all domains). Show me how to sort an array in parallel on _ANY_ hardware (CPUs, GPUs, FPGAs) with OpenMP. With C++ is as simple as: std::sort(std::execution::par, array.begin(), array.end()); If you have a GPU, this offloads to the GPU. If you have an FPGA, this offloads to the FPGA. If you have a CPU with 200 cores, this uses those 200 cores. There is no need to turn your ISO C++ compliant program or libraries into OpenMP. That prevents them from being used by many domains on which C++ runs on. It also adds an external dependency for parallelism, for no good reason. For any problem that OpenMP can solve, OpenMP is _always_ a worse solution than just using strictly ISO standard and compliant C++. OpenMP has completely lost a reason to exist. It's not 1990 anymore.
  - imtringued 4y ago
    You sure know how to ruin a good thing with bad demeanor. When someone likes something and you say they shouldn't like it because it does exactly the same thing in a different way, you are actively driving people away from the better thing.
    - maxwell86 4y ago
      It's ok to like OpenMP. What I disagree with is that it should be suggested to beginners as the way to parallelize their C++ programs. That's like telling a Javascript programmer that they should parallelize their programs by using Python or C. Show them how to do it in Javascript, or in this case, in C++, so that they don't have to learn a whole new programming model or language to just write parallel code. Particularly when C++ has supported this for so long now.
      - dragontamer 4y ago
        OpenMP is a set of #pragma that just sit in your C++ code directly. > What I disagree with is that it should be suggested to beginners as the way to parallelize their C++ programs. I guess we can agree to disagree then. If beginners think your way is easier, they're welcome to try. But there's plenty of production code examples that show the simplicity of OpenMP.
  - dragontamer 4y ago
    #pragma openmp target parallel for The "target" now makes the for-loop discussed a GPU or FPGA algorithm. Now what's strange about this is... you seemed to have known this already? So I've had difficulty making an actual response to you. OpenMP is just one tool in my toolbox. To be honest, I've found it to be not flexible enough for most of my usage, but its gross simplicity is again, one of the easiest C++ / C tools I've ever used. Yes, even for playing or dabbling in GPGPU programming. Furthermore, OpenMP is usable on GCC, Clang. Its even available (OpenMP2.0 at least) on MSVC++ (though OMP 2.0 leaves much to be desired, that's still enough for some degree of programming on Windows). So OpenMP code on say, Blender (3d raytracing program) runs on pretty much all important C++ platforms. GPU and FPGA programming is complicated to actually perform well, because GPUs and FPGAs have a huge PCIe 3.0 bottleneck. A lot of code in CPU-land can stay in L1, L2, or L3 cache and outperform the PCIe-transfer alone. In contrast, CPU-to-CPU transfers are very quick (and exist on the L3 to L3 transfer or L2 to L2 transfer speeds), so your "cost of communication" is very low. I don't want to discourage any beginner from playing with GPU code (especially if they're "just messing around"). GPU code is easier to write than most expect. But its surprisingly difficult to actually beat CPU code with GPU-offload code. If its not something that works out for you, that's fine I guess? There's a lot of different tools for a lot of different situations. -------- A fun OMP thing btw, is... #pragma openmp parallel for simd Which (tries to) compile your program into SIMD code, like AVX512 or NEON for ARM. OMP isn't as flexible as writing your own threads by hand, but its easy to experiment with many forms of parallelism with the same code. There's also nifty attributes, like "firstprivate" or "collapse", or "reduction" clause... as well as having different schedulers (static, dynamic, guided). Honestly, its really good for prototyping. You write one for-loop, but have all these knobs and dials to try out a bunch of different strategies. But for "final code", hand-crafted threaded code really can't be beaten. -------- BTW: I don't think that C++ platforms like NVidia nvcc or AMD's ROCm support for_each_n. And even if they did, that's not how you really write GPU-parallelism programs.
gumby 4y ago
> If you are using C++, and want to parallelize something, just add "std::execution::par" to your algorithms. Do any of the shipping standard libraries actually implement execution policies? I only use gcc and clang so have to resort to TBB to get this capability.
- phkahler 4y ago
  >> Do any of the shipping standard libraries actually implement execution policies? I only use gcc and clang so have to resort to TBB to get this capability. Looks like it's C++17 and C++20 feature: https://en.cppreference.com/w/cpp/algorithm/execution_policy_tag https://en.cppreference.com/w/cpp/algorithm/execution_policy...
phkahler 4y ago
OpenMP code can be compiled as single threaded on compilers that don't support it without code changes. It's not a language but more like a set of annotations to be added. I was not aware of C++ having something similar. Is that a new feature? Edit: YES it's C++17 and later: https://en.cppreference.com/w/cpp/algorithm/execution_policy_tag https://en.cppreference.com/w/cpp/algorithm/execution_policy...

dragontamer 4y ago

OpenMP can parellize a for loop like: #pragma openmp parallel for for(int i=0; i<1000000; i++) C[i] = A[i] + B[i]; It's normal C++ and the pragma auto-parallelizes the loop. It's actually really easy and convenient. OpenMP is probably the easiest join/fork model of parallelism on any C++ system I've used. It doesn't always get the best utilization of your CPU cores, but it's really, really simple. It's the best way to start IMO, far easier than std::thread, or switching to functional style for other libraries. Just write the same code as before but with a few #pragma omp statements here and there.

maxwell86 4y ago
> It's normal C++ C++ is an ISO standard, you can use it _everywhere_, in space, in automotive, in aviation, in trains, in medical devices, _everywhere_. OpenMP is not C++, it is a different programming language than C++. OpenMP is not an ISO standard, you can't use it in _most_ domains that you can use C++. Your example: #pragma openmp parallel for for(int i=0; i<1000000; i++) C[i] = A[i] + B[i]; shows how bad OpenMP is. It does not run in parallel on GPUs or on FPGAS (lacking target offload directives), and you can't use it on most domains in which you can use C++. The following is ISO standard C++: std::for_each_n(std::execution::par, std::ranges::iota(0).begin(), 1000000, [](int i) { C[i] = A[i] + B[i]; }); it runs in _parallel_ EVERYWHERE: GPUs, CPUs, FPGAS, and it is certified for all domains for which C++ is (that is: all domains). Show me how to sort an array in parallel on _ANY_ hardware (CPUs, GPUs, FPGAs) with OpenMP. With C++ is as simple as: std::sort(std::execution::par, array.begin(), array.end()); If you have a GPU, this offloads to the GPU. If you have an FPGA, this offloads to the FPGA. If you have a CPU with 200 cores, this uses those 200 cores. There is no need to turn your ISO C++ compliant program or libraries into OpenMP. That prevents them from being used by many domains on which C++ runs on. It also adds an external dependency for parallelism, for no good reason. For any problem that OpenMP can solve, OpenMP is _always_ a worse solution than just using strictly ISO standard and compliant C++. OpenMP has completely lost a reason to exist. It's not 1990 anymore.
- imtringued 4y ago
  You sure know how to ruin a good thing with bad demeanor. When someone likes something and you say they shouldn't like it because it does exactly the same thing in a different way, you are actively driving people away from the better thing.
  - maxwell86 4y ago
    It's ok to like OpenMP. What I disagree with is that it should be suggested to beginners as the way to parallelize their C++ programs. That's like telling a Javascript programmer that they should parallelize their programs by using Python or C. Show them how to do it in Javascript, or in this case, in C++, so that they don't have to learn a whole new programming model or language to just write parallel code. Particularly when C++ has supported this for so long now.
    - dragontamer 4y ago
      OpenMP is a set of #pragma that just sit in your C++ code directly. > What I disagree with is that it should be suggested to beginners as the way to parallelize their C++ programs. I guess we can agree to disagree then. If beginners think your way is easier, they're welcome to try. But there's plenty of production code examples that show the simplicity of OpenMP.
- dragontamer 4y ago
  #pragma openmp target parallel for The "target" now makes the for-loop discussed a GPU or FPGA algorithm. Now what's strange about this is... you seemed to have known this already? So I've had difficulty making an actual response to you. OpenMP is just one tool in my toolbox. To be honest, I've found it to be not flexible enough for most of my usage, but its gross simplicity is again, one of the easiest C++ / C tools I've ever used. Yes, even for playing or dabbling in GPGPU programming. Furthermore, OpenMP is usable on GCC, Clang. Its even available (OpenMP2.0 at least) on MSVC++ (though OMP 2.0 leaves much to be desired, that's still enough for some degree of programming on Windows). So OpenMP code on say, Blender (3d raytracing program) runs on pretty much all important C++ platforms. GPU and FPGA programming is complicated to actually perform well, because GPUs and FPGAs have a huge PCIe 3.0 bottleneck. A lot of code in CPU-land can stay in L1, L2, or L3 cache and outperform the PCIe-transfer alone. In contrast, CPU-to-CPU transfers are very quick (and exist on the L3 to L3 transfer or L2 to L2 transfer speeds), so your "cost of communication" is very low. I don't want to discourage any beginner from playing with GPU code (especially if they're "just messing around"). GPU code is easier to write than most expect. But its surprisingly difficult to actually beat CPU code with GPU-offload code. If its not something that works out for you, that's fine I guess? There's a lot of different tools for a lot of different situations. -------- A fun OMP thing btw, is... #pragma openmp parallel for simd Which (tries to) compile your program into SIMD code, like AVX512 or NEON for ARM. OMP isn't as flexible as writing your own threads by hand, but its easy to experiment with many forms of parallelism with the same code. There's also nifty attributes, like "firstprivate" or "collapse", or "reduction" clause... as well as having different schedulers (static, dynamic, guided). Honestly, its really good for prototyping. You write one for-loop, but have all these knobs and dials to try out a bunch of different strategies. But for "final code", hand-crafted threaded code really can't be beaten. -------- BTW: I don't think that C++ platforms like NVidia nvcc or AMD's ROCm support for_each_n. And even if they did, that's not how you really write GPU-parallelism programs.

gumby 4y ago

> If you are using C++, and want to parallelize something, just add "std::execution::par" to your algorithms. Do any of the shipping standard libraries actually implement execution policies? I only use gcc and clang so have to resort to TBB to get this capability.

phkahler 4y ago
>> Do any of the shipping standard libraries actually implement execution policies? I only use gcc and clang so have to resort to TBB to get this capability. Looks like it's C++17 and C++20 feature: https://en.cppreference.com/w/cpp/algorithm/execution_policy_tag https://en.cppreference.com/w/cpp/algorithm/execution_policy...

phkahler 4y ago

OpenMP code can be compiled as single threaded on compilers that don't support it without code changes. It's not a language but more like a set of annotations to be added. I was not aware of C++ having something similar. Is that a new feature? Edit: YES it's C++17 and later: https://en.cppreference.com/w/cpp/algorithm/execution_policy_tag https://en.cppreference.com/w/cpp/algorithm/execution_policy...