GPU
The tutorials in amrex-tutorials/ExampleCodes/GPU
demonstrate the implementation
of AMReX’s GPU toolkit as well as provide GPU ported versions of CPU
tutorials to help applications convert to GPUs.
Your first AMReX GPU application
This is a step-by-step guide to preparing, compiling and running your first
AMReX GPU program. This guide will use Tutorials/GPU/Launch
,
and instructions will focus on ORNL’s systems:
Before compiling, the
pgi
andcuda
software must be available. On ORNL systems, the modules can be loaded directly by typing:
module load pgi cuda
Go to
Tutorials/GPU/Launch
to compile the executable. Compile withmake USE_CUDA=TRUE COMP=pgi USE_MPI=TRUE USE_OMP=FALSE
, or edit theGNUmakefile
to match this configuration and runmake
. This should result in an executable:main3d.pgi.MPI.CUDA.ex
.On Summit systems, this executable can be submitted by using one of the run scripts found in
Tutorials/GPU
.run.script
can be used to run on Summitdev, andrun.summit
can be used for Summit. To change the number of ranks and GPUs used in the simulation, change the number of resource sets,n
in thejsrun
line. For the firstLaunch
tutorial, usen=1
and setINPUTS=""
because no input file is used in this example.
When ready, submit the job script (on Summit: bsub run.script
).
Congratulations! You’ve accelerated AMReX using GPUs!
Launch
Launch shows multiple examples of how GPU work can be offloaded using the tools available in AMReX. It includes examples of multiple AMReX macro launch methods, launching a Fortran function using CUDA and launching work using OpenACC and OpenMP offloading. This tutorial will be regularly updated with AMReX’s preferred GPU launch methodologies.
CNS
CNS is a direct GPU port of the ExampleCodes/EB/CNS
example.
AmrCore
AmrCore is a direct GPU port of the ExampleCodes/Amr/Advection_AmrCore
tutorial
that advects a single scalar field with a velocity field specified on faces, using
strategies similar to HeatEquation and CNS.