GPU

In this chapter, we will present GPU support in AMReX. AMReX targets NVIDIA, AMD, and Intel GPUs using their native vendor languages and therefore requires CUDA, HIP/ROCm, and SYCL for NVIDIA, AMD, and Intel GPUs, respectively. Users can also use OpenMP and/or OpenACC in their applications if desired.

AMReX supports NVIDIA GPUs with compute capability >= 6 and CUDA >= 11, and AMD GPUs with ROCm >= 5. While SYCL compilers are still under development in preparation for Aurora, AMReX officially supports only the latest publicly released version of the oneAPI compiler.

For complete details of CUDA, HIP, SYCL, OpenMP, and OpenACC languages, see their respective documentation.

Note that this documentation is currently focused on CUDA and HIP. SYCL documentation is forthcoming.

A number of tutorials can be found at Tutorials/GPU.

Contents:

• Overview of AMReX GPU Support
• Building GPU Support
• Gpu Namespace and Macros
• Memory Allocation
• GPU Safe Classes and Functions
• Kernel Launch
• Stream and Synchronization
• External GPU Streams
• An Example of Migrating to GPU
• Assertions and Error Checking
• Particle Support
• Profiling with GPUs
• Performance Tips
• Inputs Parameters