The tutorials in
amrex-tutorials/ExampleCodes/GPU demonstrate the implementation
of AMReX’s GPU toolkit as well as provide GPU ported versions of CPU
tutorials to help applications convert to GPUs.
Your first AMReX GPU application
This is a step-by-step guide to preparing, compiling and running your first
AMReX GPU program. This guide will use
and instructions will focus on ORNL’s systems:
Before compiling, the
cudasoftware must be available. On ORNL systems, the modules can be loaded directly by typing:
module load pgi cuda
Tutorials/GPU/Launchto compile the executable. Compile with
make USE_CUDA=TRUE COMP=pgi USE_MPI=TRUE USE_OMP=FALSE, or edit the
GNUmakefileto match this configuration and run
make. This should result in an executable:
On Summit systems, this executable can be submitted by using one of the run scripts found in
run.scriptcan be used to run on Summitdev, and
run.summitcan be used for Summit. To change the number of ranks and GPUs used in the simulation, change the number of resource sets,
jsrunline. For the first
INPUTS=""because no input file is used in this example.
When ready, submit the job script (on Summit:
Congratulations! You’ve accelerated AMReX using GPUs!
Launch shows multiple examples of how GPU work can be offloaded using the tools available in AMReX. It includes examples of multiple AMReX macro launch methods, launching a Fortran function using CUDA and launching work using OpenACC and OpenMP offloading. This tutorial will be regularly updated with AMReX’s preferred GPU launch methodologies.
CNS is a direct GPU port of the
AmrCore is a direct GPU port of the
that advects a single scalar field with a velocity field specified on faces, using
strategies similar to HeatEquation and CNS.