7 ms·

Porting HPC Applications to AMD Instinct MI300A Using Unified Memory and OpenMP

95 points by arcanus 2y ago

bee_rider 2y ago
APU’s for HPC are going to be a wild ride. Accelerated computing in shared memory. Get CPU-focused folks will actually get access to some high throughput compute accessible on the sort of timescales that we can actually reason about (the GPU is so far away).
- djmips 2y ago
  All video game consoles use APUs and it does make memory related operations potentially faster but at least for video games it's not the bottleneck. I suppose for HPC it might have more significance.
  - bayindirh 2y ago
    If you're doing simulations, or poking big matrices continuously on CPUs, you can saturate the memory controller pretty easily. If you know what you're doing, your FPU or vector units are saturated at the same time, so "whole system" becomes the bottleneck while it tries to keep itself cool. Games move that kind of data in the beginning and doesn't stream new data that much after the initial texture and model data. If you are working on HPC with GPUs, you may need to constantly stream in new data to the GPU while streaming out the results. This is why datacenter/compute GPUs have multiple independent DMA engines.
  - crest 2y ago
    Afaik those unified memory architectures are mostly neither cache coherent nor do they support virtual addresses efficiently (you have to trap into privileged code to pin/unpin the mappings) which means that the relative cost is lower than a dedicated GPU accessed via PCIe slots, but still to high. Only the "boring" old Bobcat based AMD APUs supported accessing unpinned virtual memory from the L3 (aka system level) cache and nobody bothered with porting code to them.
    - paulmd 2y ago
      > Afaik those unified memory architectures are mostly neither cache coherent nor do they support virtual addresses efficiently (you have to trap into privileged code to pin/unpin the mappings) which means that the relative cost is lower than a dedicated GPU accessed via PCIe slots, but still to high. Only the "boring" old Bobcat based AMD APUs supported accessing unpinned virtual memory from the L3 (aka system level) cache and nobody bothered with porting code to them. Other way around, bobcat was the era of “onion bus”/“garlic bus” and today things like apple silicon don’t need to be explicitly accessed in certain ways afaik. https://www.realworldtech.com/fusion-llano/3/ https://www.realworldtech.com/fusion-llano/3/ https://www.anandtech.com/show/16226/apple-silicon-m1-a14-deep-dive/2 https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de...
- JonChesterfield 2y ago
  APUs are very cool for GPU programming in general. Explicitly copying data to/from GPUs is a definite nuisance. I'm hopeful that the MI300A will have a positive knock on effect on the low power APUs in laptops and similar.
  - imtringued 2y ago
    >Explicitly copying data to/from GPUs is a definite nuisance. CXL allows fine grained shared memory, but people look at the shiny high bandwidth NVLink and talk about how much better it is for... AI.
curt15 2y ago
I was talking with a friend in HPC lately who said that AMD is actually quite competitive in the HPC space these days. For example, Frontier (https://docs.olcf.ornl.gov/systems/frontier_user_guide.html https://docs.olcf.ornl.gov/systems/frontier_user_guide.html) is an all-AMD installation. Do scientists actually use ROCm in their code or does AMD have another programming framework for their Instinct chips?
- kkielhofner 2y ago
  I currently have a project with ORNL OLCF (on Frontier). The short answer is yes. Happy to answer any questions I can.
  - ysleepy 2y ago
    ROCm or HIP? Does it start out with porting a lot from CUDA etc. or starting fresh on top of the AMD APIs? How much of the project time is spent on that compute API stuff in comparison to "payload" work?
    - kkielhofner 2y ago
      My project is ROCm (torch, more or less) and working with OLCF staff I've never heard of HIP in use but based on their training series it is supported[0]. Of course my personal experience isn't exhaustive and it can be inferred from the ongoing training series that it is in use in some cases. Speaking from personal experience ROCm itself is... Challenging (which I already knew from prior endeavors). We've taken to dev and staging workloads on more typical MI2xx hardware and then working it over to Frontier. We currently have 20k node hours on Frontier via a Director's Discretion Project[1]. It's a relatively simple application and at the end of the day you have access to significant compute so depending on workload the extra effort for ROCm, etc is still worth it. [0] - https://www.olcf.ornl.gov/hip-training-series/ https://www.olcf.ornl.gov/hip-training-series/ [1] - https://www.olcf.ornl.gov/for-users/documents-forms/olcf-directors-discretion-project-application/ https://www.olcf.ornl.gov/for-users/documents-forms/olcf-dir...
    - KeplerBoy 2y ago
mathiasgredal 2y ago
Having looked briefly at the code I still think C++17 parallel algorithms are more ergonomic compared to OpenMP: https://rocm.blogs.amd.com/software-tools-optimization/hipstdpar/README.html https://rocm.blogs.amd.com/software-tools-optimization/hipst...
- mgaunard 2y ago
  funny how we only get LoC between the different versions, but not the performance... Of course the parallel algorithms are shorter, it's a more high-level interface. But being explicit gives you more control and potentially more performance.
- bee_rider 2y ago
  Is language support why people like OpenMP? I think it is nice because it supports both C and Fortran, and they use the same runtime, so you can do things like pin threads to cores or avoid oversubscription. Stuff like calling a Fortran library that uses OpenMP, from a C code that also uses OpenMP, doesn’t require anything clever.
  - jltsiren 2y ago
    OpenMP has been around for a long time. People know how to use it, and it has gained many features that are useful for scientific computing. The consortium behind OpenMP consists mostly of hardware companies and organizations doing scientific computing. Software companies are largely missing. That may contribute to the popularity of OpenMP, as the interests of scientific computing and software development are often different.
  - phkahler 2y ago
    >> Is language support why people like OpenMP? I use it sometimes with C++ because it is super easy to make "embarrassingly parallel" code actually run in parallel. And by using nothing but #pragma statements it will still compile single threaded if you don't have OMP as the pragmas will be ignored.
Agingcoder 2y ago
I’ve been waiting for something like that in the HPC space for years - that’s what I wanted when HSA first came out. https://en.wikipedia.org/wiki/Heterogeneous_System_Architecture https://en.wikipedia.org/wiki/Heterogeneous_System_Architect...

95 points by arcanus 2y ago

APU’s for HPC are going to be a wild ride. Accelerated computing in shared memory. Get CPU-focused folks will actually get access to some high throughput compute accessible on the sort of timescales that we can actually reason about (the GPU is so far away).

djmips 2y ago
All video game consoles use APUs and it does make memory related operations potentially faster but at least for video games it's not the bottleneck. I suppose for HPC it might have more significance.
- bayindirh 2y ago
  If you're doing simulations, or poking big matrices continuously on CPUs, you can saturate the memory controller pretty easily. If you know what you're doing, your FPU or vector units are saturated at the same time, so "whole system" becomes the bottleneck while it tries to keep itself cool. Games move that kind of data in the beginning and doesn't stream new data that much after the initial texture and model data. If you are working on HPC with GPUs, you may need to constantly stream in new data to the GPU while streaming out the results. This is why datacenter/compute GPUs have multiple independent DMA engines.
- crest 2y ago
  Afaik those unified memory architectures are mostly neither cache coherent nor do they support virtual addresses efficiently (you have to trap into privileged code to pin/unpin the mappings) which means that the relative cost is lower than a dedicated GPU accessed via PCIe slots, but still to high. Only the "boring" old Bobcat based AMD APUs supported accessing unpinned virtual memory from the L3 (aka system level) cache and nobody bothered with porting code to them.
  - paulmd 2y ago
    > Afaik those unified memory architectures are mostly neither cache coherent nor do they support virtual addresses efficiently (you have to trap into privileged code to pin/unpin the mappings) which means that the relative cost is lower than a dedicated GPU accessed via PCIe slots, but still to high. Only the "boring" old Bobcat based AMD APUs supported accessing unpinned virtual memory from the L3 (aka system level) cache and nobody bothered with porting code to them. Other way around, bobcat was the era of “onion bus”/“garlic bus” and today things like apple silicon don’t need to be explicitly accessed in certain ways afaik. https://www.realworldtech.com/fusion-llano/3/ https://www.realworldtech.com/fusion-llano/3/ https://www.anandtech.com/show/16226/apple-silicon-m1-a14-deep-dive/2 https://www.anandtech.com/show/16226/apple-silicon-m1-a14-de...
JonChesterfield 2y ago
APUs are very cool for GPU programming in general. Explicitly copying data to/from GPUs is a definite nuisance. I'm hopeful that the MI300A will have a positive knock on effect on the low power APUs in laptops and similar.
- imtringued 2y ago
  >Explicitly copying data to/from GPUs is a definite nuisance. CXL allows fine grained shared memory, but people look at the shiny high bandwidth NVLink and talk about how much better it is for... AI.

curt15 2y ago

I was talking with a friend in HPC lately who said that AMD is actually quite competitive in the HPC space these days. For example, Frontier (https://docs.olcf.ornl.gov/systems/frontier_user_guide.html https://docs.olcf.ornl.gov/systems/frontier_user_guide.html) is an all-AMD installation. Do scientists actually use ROCm in their code or does AMD have another programming framework for their Instinct chips?

kkielhofner 2y ago
I currently have a project with ORNL OLCF (on Frontier). The short answer is yes. Happy to answer any questions I can.
- ysleepy 2y ago
  ROCm or HIP? Does it start out with porting a lot from CUDA etc. or starting fresh on top of the AMD APIs? How much of the project time is spent on that compute API stuff in comparison to "payload" work?
  - kkielhofner 2y ago
    My project is ROCm (torch, more or less) and working with OLCF staff I've never heard of HIP in use but based on their training series it is supported[0]. Of course my personal experience isn't exhaustive and it can be inferred from the ongoing training series that it is in use in some cases. Speaking from personal experience ROCm itself is... Challenging (which I already knew from prior endeavors). We've taken to dev and staging workloads on more typical MI2xx hardware and then working it over to Frontier. We currently have 20k node hours on Frontier via a Director's Discretion Project[1]. It's a relatively simple application and at the end of the day you have access to significant compute so depending on workload the extra effort for ROCm, etc is still worth it. [0] - https://www.olcf.ornl.gov/hip-training-series/ https://www.olcf.ornl.gov/hip-training-series/ [1] - https://www.olcf.ornl.gov/for-users/documents-forms/olcf-directors-discretion-project-application/ https://www.olcf.ornl.gov/for-users/documents-forms/olcf-dir...
  - KeplerBoy 2y ago

mathiasgredal 2y ago

Having looked briefly at the code I still think C++17 parallel algorithms are more ergonomic compared to OpenMP: https://rocm.blogs.amd.com/software-tools-optimization/hipstdpar/README.html https://rocm.blogs.amd.com/software-tools-optimization/hipst...

mgaunard 2y ago
funny how we only get LoC between the different versions, but not the performance... Of course the parallel algorithms are shorter, it's a more high-level interface. But being explicit gives you more control and potentially more performance.
bee_rider 2y ago
Is language support why people like OpenMP? I think it is nice because it supports both C and Fortran, and they use the same runtime, so you can do things like pin threads to cores or avoid oversubscription. Stuff like calling a Fortran library that uses OpenMP, from a C code that also uses OpenMP, doesn’t require anything clever.
- jltsiren 2y ago
  OpenMP has been around for a long time. People know how to use it, and it has gained many features that are useful for scientific computing. The consortium behind OpenMP consists mostly of hardware companies and organizations doing scientific computing. Software companies are largely missing. That may contribute to the popularity of OpenMP, as the interests of scientific computing and software development are often different.
- phkahler 2y ago
  >> Is language support why people like OpenMP? I use it sometimes with C++ because it is super easy to make "embarrassingly parallel" code actually run in parallel. And by using nothing but #pragma statements it will still compile single threaded if you don't have OMP as the pragmas will be ignored.

Agingcoder 2y ago

I’ve been waiting for something like that in the HPC space for years - that’s what I wanted when HSA first came out. https://en.wikipedia.org/wiki/Heterogeneous_System_Architecture https://en.wikipedia.org/wiki/Heterogeneous_System_Architect...