Runtime Parameters

This chapter contains a list of AMReX ParmParse runtime parameters and their default values. They can be set by either including them in an inputs file, or specifying them at the command line, or passing a function to amrex::Initialize and the function adds parameters to AMReX’s ParmParse’s parameter database. For more information on ParmParse, see ParmParse.

Important

AMReX reserves the following prefixes in ParmParse parameters: amr, amrex, blprofiler, device, DistributionMapping, eb2, fab, fabarray, geometry, integration, particles, tiny_profiler, and vismf.

AMR

AMReX applications with AMR use either class AmrCore or the more specialized class Amr. Since class Amr is derived from class AmrCore, the parameters for the AmrCore class also apply to the Amr class. Additionally, class AmrCore is derived from class AmrMesh, so AmrMesh member functions are also available to AmrCore and Amr.

AmrCore Class

Below are a list of important ParmParse parameters. However, AMReX applications can choose to avoid them entirely by use this AMRCore constructor AmrCore(Geometry const& level_0_geom, AmrInfo const& amr_info), where struct AmrInfo contains all the information that can be set via ParmParse.

amr.verbose: int = 0: This controls the verbosity level of AmrCore functions.

amr.n_cell: int array = [none]: This parameter is used only when n_cell is not provided as an argument to AmrCore constructors. It specifies the number of cells in each dimension on Level 0.

amr.max_level: int = [none]: This parameter is used only when max_level is not provided as an argument to AmrCore constructors. It specifies the maximum level of refinement allowed. Note that the total number of levels, including the base level 0, is max_level+1.

amr.ref_ratio: int array = 2 2 2 ... 2: If the refinement ratio is not provided as an argument to AmrCore constructors and amr.ref_ratio_vect is not found in the ParmParse database, this parameter will be used to set the refinement ratios between AMR levels. If there are more AMR levels than the size of the integer parameter array, the last integer will be used as the refinement ratio for the unspecified levels. For example, if max_level is 4 and the provided amr.ref_ratio parameter is 2 4, the refinement ratios are 2, 4, 4 and 4, for levels 0/1, 1/2, 2/3 and 3/4, respectively.

amr.ref_ratio_vect: int array = [none]: If the refinement ratio is not provided as an argument to AmrCore constructors and amr.ref_ratio_vect is found in the ParmParse database, it will be used to set the refinement ratios between AMR levels. It’s an error if the size of the integer array, if found, is less than max_level*AMREX_SPACEDIM. The first AMREX_SPACEDIM numbers specify the refinement ratios in the AMREX_SPACEDIM dimensions between levels 0 and 1, the next AMREX_SPACEDIM numbers specify the ratios for levels 1 and 2, and so on.

amr.max_grid_size: int array = [build dependent]: This controls the maximum grid size on AMR levels, one value for each level. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels. The default value is 128 for 1D and 2D runs. For 3D runs, the default value is 64 and 32, for GPU and CPU runs, respectively. Note that the user can also call AmrMesh::SetMaxGridSize to set the maximum grid sizes. Additionally, the values set by this parameter can be overridden by amr.max_grid_size_x, amr.max_grid_size_y and amr.max_grid_size_z.

amr.max_grid_size_x: int array = [none]: If provided, this will override the maximum grid size in the x-direction set by amr.max_grid_size. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels.

amr.max_grid_size_y: int array = [none]: If provided, this will override the maximum grid size in the y-direction set by amr.max_grid_size. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels.

amr.max_grid_size_z: int array = [none]: If provided, this will override the maximum grid size in the z-direction set by amr.max_grid_size. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels.

amr.blocking_factor: int array = [build dependent]: This controls the blocking factor on AMR levels, one value for each level. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels. The default value is 8. Note that the user can also call AmrMesh::SetBlockingFactor to set the blocking factors. Additionally, the values set by this parameter can be overridden by amr.blocking_factor_x, amr.blocking_factor_y and amr.blocking_factor_z.

amr.blocking_factor_x: int array = [none]: If provided, this will override the blocking factor in the x-direction set by amr.blocking_factor. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels.

amr.blocking_factor_y: int array = [none]: If provided, this will override the blocking factor in the y-direction set by amr.blocking_factor. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels.

amr.blocking_factor_z: int array = [none]: If provided, this will override the blocking factor in the z-direction set by amr.blocking_factor. If the size of the integer array is less than the total number of levels, the last integer will be used for the unspecified levels.

amr.n_proper: int = 1: This parameter controls the proper nesting of grids on AMR levels. For example, if we have blocking_factor = 8, ref_ratio = 2 and n_proper = 1, there will be at least 8/2*1 = 4 coarse level cells outside the fine level grids except at the physical boundaries. Note that the user can also call AmrMesh::SetNProper(int) to set the proper nesting parameter.

amr.grid_eff: amrex::Real = 0.7: This parameter controls the grid efficiency threshold during grid creation. While a higher value can enhance efficiency, it may negatively impact overall performance, especially for GPU runs, because it tends to create smaller grids. Note that the user can also call AmrMesh::SetGridEff(Real) to set the grid efficiency threshold.

amr.n_error_buf: int array = 1 1 1 ... 1: This parameter controls how many extra cells will be tagged around every tagged cell. For example, if n_error_buf = 2, tagging cell (i,j,k) will result in the tagging of the region of from lower corner (i-2,j-2,k-2) to upper corner (i+2,j+2,k+2). If the size of the integer array is less than the number of levels, the last integer will be used for the unspecified levels. Note that the values set by this parameter can be overridden by amr.n_error_buf_x, amr.n_error_buf_y and amr.n_error_buf_z.

amr.n_error_buf_x: int array = [none]: This parameter controls the error buffer size in the x-direction. If the size of the integer array is less than the number of levels, the last integer will be used for the unspecified levels.

amr.n_error_buf_y: int array = [none]: This parameter controls the error buffer size in the y-direction. If the size of the integer array is less than the number of levels, the last integer will be used for the unspecified levels.

amr.n_error_buf_z: This parameter controls the error buffer size in the z-direction. If the size of the integer array is less than the number of levels, the last integer will be used for the unspecified levels.

amr.refine_grid_layout: bool = true: If it’s true, AMReX will attempt to chop new grids into smaller chunks ensuring at least one grid per MPI process, provided this does not violate the blocking factor constraint.

amr.refine_grid_layout_x: bool = [none]: This parameter, if found, will override the amrex.refine_grid_layout parameter in the x-direction.

amr.refine_grid_layout_y: bool = [none]: This parameter, if found, will override the amrex.refine_grid_layout parameter in the y-direction.

amr.refine_grid_layout_z: bool = [none]: This parameter, if found, will override the amrex.refine_grid_layout parameter in the z-direction.

amr.check_input: bool = true: If this is true, AMReX will check if the various parameters in AmrMesh are reasonable.

Amr Class

Warning

These parameters are specific to class Amr based applications. If your application uses class AmrCore directly, they do not apply unless you have provided implementations for them.

Subcycling

amr.subcycling_mode: string = Auto: This controls the subcycling mode of class Amr. Possible value are None for no subcycling, or Auto for subcycling.

Regrid

amr.regrid_int: int array = 1 1 1 ... 1: This controls how often we perform the regrid operation on AMR levels 0 to max_level-1. If the parameter is a single value, it will be used on all levels. If the parameter is an array of more than one values, the size must be at least max_level and values after the first max_level elements are ignored.

amr.regrid_on_restart: bool = false: This controls whether we perform regrid immediately after restart.

amr.force_regrid_level_zero: bool = false: This controls whether we perform regrid on level 0.

amr.compute_new_dt_on_regrid: bool = false: This controls whether we re-compute dt after regrid.

amr.initial_grid_file: string = [none]: If this is set, the initial grids will be read from the specified file.

amr.regrid_file: string = [none]: If this is set, regrid will use the grids in the specified file.

I/O

amr.restart: string = [none]: If this is set, the simulation will restart from the specified checkpoint file.

amr.plotfile_on_restart: bool = false: If this is set to true, a plotfile will be written after restart.

amr.file_name_digits: int = 5: This parameter specifies the minimum number of digits in checkpoint and plotfile names.

amr.checkpoint_files_output: bool = true: This controls whether we write checkpoint files.

amr.check_file: string = chk: This sets the “root” of checkpoint file names. For example, the checkpoint files are named chk00000, chk001000, etc. by default.

amr.check_int: int = -1: This controls the interval of writing checkpoint files, defined as the number of level 0 steps between each checkpoint. A value less than 1 indicates no checkpoint files will be written.

amr.check_per: amrex::Real = -1: This controls the interval of writing checkpoint files, defined as the time (not the wall time) elapsed between each checkpoint. A value less or equal to 0 indicates no checkpoint files will be written.

amr.checkpoint_nfiles: int = 64: This is the maximum number of binary files per MultiFab when writing checkpoint files.

amr.plot_files_output: bool = true: This controls whether we write plot files.

amr.plot_file: string = plt: This sets the “root” of plot file names. For example, the plot files are named plt00000, plt001000, etc. by default.

amr.plot_int: int = -1: This controls the interval of writing plot files, defined as the number of level 0 steps between each plot file. A value less than 1 indicates no plot files will be written.

amr.plot_per: amrex::Real = -1: This controls the interval of writing plot files, defined as the time (not the wall time) elapsed between each plot file. A value less or equal to 0 indicates no plot files will be written.

amr.plot_log_per: amrex::Real = -1: This controls the interval of writing plot files, defined as the log10 time (not the wall time) elapsed between each plot file. A value less or equal to 0 indicates no plot files will be written.

amr.plot_max_level: int = amr.max_level: This controls the finest level in a plot file. For example, if the finest level in a run is 3, but this parameter is set to 1, only levels 0 and 1 will be saved in a plot file.

amr.plot_nfiles: int = 64: This is the maximum number of binary files per MultiFab when writing plot files.

amr.plot_vars: string array = [none]: If this parameter is set, the variables specified in the string array will be the state variables saved in the plot files. The special values ALL and NONE mean that all or none of the state variables will be saved. If this parameter is not set, all state variables will be saved.

amr.derive_plot_vars: string array = [none]: If this parameter is set, the variables specified in the string array will be the derive variables saved in the plot files. The special values ALL and NONE mean that all or none of the derive variables will be saved. If this parameter is not set, none of the derive variables will be saved.

amr.small_plot_file: string = smallplt: This sets the “root” of small plot file names. For example, the small plot files are named smallplt00000, smallplt001000, etc. by default.

amr.small_plot_int: int = -1: This controls the interval of writing small plot files, defined as the number of level 0 steps between each small plot file. A value less than 1 indicates no small plot files will be written.

amr.small_plot_per: amrex::Real = -1: This controls the interval of writing small plot files, defined as the time (not the wall time) elapsed between each small plot file. A value less or equal to 0 indicates no small plot files will be written.

amr.small_plot_log_per: amrex::Real = -1: This controls the interval of writing small plot files, defined as the log10 time (not the wall time) elapsed between each small plot file. A value less or equal to 0 indicates no small plot files will be written.

amr.small_plot_vars: string array = [none]: If this parameter is set, the variables specified in the string array will be the state variables saved in the small plot files. The special values ALL and NONE mean that all or none of the state variables will be saved. If this parameter is not set, none of the state variables will be saved.

amr.derive_small_plot_vars: string array = [none]: If this parameter is set, the variables specified in the string array will be the derive variables saved in the small plot files. The special values ALL and NONE mean that all or none of the derive variables will be saved. If this parameter is not set, none of the derive variables will be saved.

amr.message_int: int = 10

This controls the interval of checking messages during a run, defined as the number of level 0 steps between checks. A value less than 1 indicates no checking will be performed. A message refers to a file created by the user on the disk, where only the file name is checked, not its content. If the file name matches one of the following predefined names, appropriate actions will be taken.

dump_and_continue: Make a checkpoint file and continue running the simulation.
stop_run: Stop the simulation.
dump_and_stop: Make a checkpoint file and stop the simulation.
plot_and_continue: Make a plot file and continue running the simulation.
small_plot_and_continue: Make a small plot file and continue running the simulation.

amr.write_plotfile_with_checkpoint: bool = true: This parameter is for the message action discussed in amr.message_int. It controls whether an action will make a plot file as well when asked to make a checkpoint file.

amr.run_log: string = [none]: If this parameter is set, the run log will be enabled and this is the log file name.

amr.run_log_terse: string = [none]: If this parameter is set, the terse run log will be enabled and this is the log file name.

amr.grid_log: string = [none]: If this parameter is set, the grid log will be enabled and this is the log file name.

amr.data_log: string = [none]: If this parameter is set, the data log will be enabled and this is the log file name.

Basic Controls

amrex.verbose: int = 1: This controls the verbosity level of AMReX. Besides using ParmParse, you can also call amrex::SetVerbose(int) to set it.

amrex.init_snan: bool = [build dependent]: This controls whether MultiFab, FArrayBox, BaseFab<double|float>, PODVectors<double|float>, Gpu::DeviceVector<double|float>, etc. will be initialized to signaling NaNs at construction. The default value is true for debug builds. For non-debug builds, the default is false unless TEST=TRUE for GNU Make or AMReX_TESTING is enabled for CMake.

amrex.abort_on_unused_inputs: bool = false: If this is true and there are unused ParmParse parameters, AMReX will abort during amrex::Finalize.

amrex.parmparse.verbose: int = amrex.verbose: If this is greater than zero, unused ParmParse variables will be printed out during amrex::Finalize or ParmParse::QueryUnusedInputs. The parameter can also be set by calling amrex::ParmParse::SetVerbose(int).

amrex.device.verbose: int = 0: This controls whether AMReX prints out GPU device properties such name, vendor, total memory size, etc. This is only relevant for GPU runs.

amrex.max_gpu_streams: int = 4: This controls the number of GPU streams used by AMReX. It’s only relevant for GPU runs.

amrex.omp_threads: string = system: If OpenMP is enabled, this can be used to set the default number of threads. Possible values are system, nosmt, or an integer string. The special value nosmt can be used to avoid using threads for virtual cores (aka Hyperthreading or SMT), as is default in OpenMP, and instead only spawns threads equal to the number of physical cores in the system. For the values system and nosmt, the environment variable OMP_NUM_THREADS takes precedence. If the string can be converted to an integer, OMP_NUM_THREADS is ignored.

amrex.memory_log: string = memlog: This is the name of the memory log file when memory profiling is enabled.

Communication

amrex.use_gpu_aware_mpi: bool = false: For GPU runs, this controls the memory type used for AMReX’s communication buffers. When this is true, AMReX uses GPU device memory for communication data in MPI function calls. When this is false, the data are placed in pinned memory. Note that this flag does not enable GPU-aware MPI by itself. Enabling GPU-aware MPI is system dependent. Users should consult their system’s documentation for instructions on setting up the environment and linking to GPU-aware MPI libraries.

Distribution Mapping

DistributionMapping.verbose: int = 0: This controls the verbosity level of DistributionMapping functions.

DistributionMapping.strategy: string = SFC: This is the default DistributionMapping strategy. Possible values are SFC, KNAPSACK, ROUNDROBIN, or RRSFC. Note that the default strategy can also be set by calling DistributionMapping::strategy(DistributionMapping::Strategy).

Embedded Boundary

eb2.max_grid_size: int = 64: This parameter specifies the maximum grid size in AMReX’s internal EB database, not the user’s data.

eb2.extend_domain_face: bool = true: This controls the behavior of the embedded boundary outside the domain. If this is true, the embedded boundary outside the domain is extended perpendicularly from the domain face. Otherwise, it’s generated with the user provided implicit function. Note that this parameter can be overridden by the user when calling amrex::EB2::Build with the optional parameter bool extend_domain_face.

eb2.num_coarsen_opt: int = 0: If it is greater than 0, this parameter can speed up the EB generation. It indicates that the search for EB can be performed on grids coarsened by this factor and then the EB information details will be generated on the original grids. However, the user should be aware that setting this parameter too high could result in erroneous results. Also note that this parameter can be overridden by the user when calling amrex::EB2::Build with the optional parameter int num_coarsen_opt.

eb2.geom_type: string = [none]

There are two versions of the amrex::EB2::Build function that can be used to build EB. One version is a function template that takes a user provided GeometryShop, while the other uses ParmParse parameters to build EB. For the latter version, this parameter specifies the type of the EB. Possible values include the following.

all_regular: The entire domain is regular without any EB objects.
parser: The embedded boundary is describe by eb2.parser_function.
stl: The embedded boundary will be built using an STL file specified by eb2.stl_file.

eb2.parser_function: string = [none]: When eb2.geom_type = parser, this parameter is a parser function string that contains a math expression describing the surface of the EB.

See also

Section Parser.

eb2.stl_file: string = [none]: When eb2.geom_type = stl, this is a required string parameter specifying the STL file name.

eb2.stl_scale: amrex:Real = 1: When building EB using STL, the triangles in the STL file will be scaled by the given value of this optional parameter.

eb2.stl_center: amrex::Real array = 0 0 0: When building EB using STL, this optional parameter specifies the shifted center. The original coordinates in the STL file will be shifted by the provided values.

eb2.stl_reverse_normal: bool = false: When building EB using STL, the normal direction of the triangles in the STL file will be reversed if this optional parameter is set to true.

eb2.small_volfrac: amrex::Real = [depend on the type of amrex::Real]: This parameter specifies the threshold for small cells that will be converted to covered cells. The default value is 1.e-14 if amrex::Real is double, or 1.e-5 if amrex::Real is float.

eb2.cover_multiple_cuts: bool = false: If this parameter is set to true, multi-cut cells will be converted to covered cells.

Tip

Because AMReX currently does not support multi-cut cells, it would be a runtime error if multi-cut cells are left unfixed.

eb2.maxiter: int = 32: Fixing small and multi-cut cells is an iterative process. This parameter specifies the maximum number of iterations for the fix-up process.

Error Handling

By default AMReX installs a signal handler that will be run when a signal such as segfault is received. You can also enable floating point exception trapping. The signal handler will print out backtraces that can be useful for debugging.

Note

Floating point exception trapping is not enabled by default, because compilers might generate optimized SIMD code that raises the exceptions.

amrex.signal_handling: bool = true: This controls whether AMReX should handle signals.

amrex.handle_sigsegv: bool = true: If both this flag and amrex.signal_handling are true, SIGSEGV will be handled by AMReX.

amrex.handle_sigterm: bool = false: If both this flag and amrex.signal_handling are true, SIGTERM will be handled by AMReX. This flag is false by default because this could generate lots of backtrace files on some batch systems that issue SIGTERM for jobs running out of wall clock time.

amrex.handle_sigint: bool = true: If both this flag and amrex.signal_handling are true, SIGINT will be handled by AMReX.

amrex.handle_sigabrt: bool = true: If both this flag and amrex.signal_handling are true, SIGABGT will be handled by AMReX.

amrex.handle_sigfpe: bool = true: If both this flag and amrex.signal_handling are true, SIGFPE will be handled by AMReX.

See also

Use amrex.fpe_trap_invalid, amrex.fpe_trap_zero and amrex.fpe_trap_overflow to enable FE_INVALID, FE_DIVBYZERO and FE_OVERFLOW trapping, respectively.

amrex.handle_sigill: bool = true: If both this flag and amrex.signal_handling are true, SIGILL will be handled by AMReX.

amrex.throw_exception: bool = false: If this flag is true and amrex.signal_handling is false, amrex::Abort and amrex::Error will throw std::runtime_error instead of aborting immediately. Note that according the C++ standard, if an exception is thrown and not caught, std::terminate will be called.

amrex.fpe_trap_invalid: bool = false: If SIGFPE is handled by AMReX and this flag is true, FE_INVALID (e.g., 0/0) trapping will be enabled. This flag has no effect on Windows.

amrex.fpe_trap_zero: bool = false: If SIGFPE is handled by AMReX and this flag is true, FE_DIVBYZERO (e.g., 1/0) trapping will be enabled. This flag has no effect on Windows.

amrex.fpe_trap_overflow: bool = false: If SIGFPE is handled by AMReX and this flag is true, FE_OVERFLOW (i.e., the result is too large to be representable) trapping will be enabled. This flag has no effect on Windows.

Extern

Hypre

These parameters are relevant only when Hypre support is enabled.

amrex.init_hypre: bool = true: This controls whether AMReX should call HYPRE_Init() during amrex::Initialize.

amrex.hypre_spgemm_use_vendor: bool = false: This controls whether HYPRE should use the vendor’s SpGemm functionality.

amrex.hypre_spmv_use_vendor: bool = false: This controls whether HYPRE should use the vendor’s SpMV functionality.

amrex.hypre_sptrans_use_vendor: bool = false: This controls whether HYPRE should use the vendor’s SpTrans functionality.

Geometry

All these parameters are optional for constructing a Geometry object. They are only used if the information is not provided via function arguments.

geometry.coord_sys: int = 0: This specifies the coordinate system type with valid values being 0 (Cartesian), or 1 (cylindrical), or 2 (spherical).

geometry.prob_lo: amrex::Real array = 0 0 0: This specifies the position of the lower corner of the physical domain.

geometry.prob_hi: amrex::Real array = [none]: This specifies the position of the upper corner of the physical domain. If this is provided, geometry.prob_extent will be ignored.

geometry.prob_extent: amrex::Real array = [none]: This specifies the length of the physical domain. If geometry.prob_hi is provided, this will be ignored.

geometry.is_periodic: int array = 0 0 0: These integer parameters are boolean flags to indicate whether the domain is periodic in each direction. It’s considered true (i.e., periodic) if its value is non-zero, and false (i.e., non-periodic) if its value is zero.

I/O

amrex.async_out: bool = false: If this is true, AMReX’s native mesh and particle plotfiles will be written asynchronously by a background thread.

amrex.async_out_nfiles: into = 64: This is the maximum number of binary files on each AMR level that will be used when AMReX writes a plotfile asynchronously.

vismf.verbose: int = 0: This controls the verbosity level of VisMF functions.

Memory

amrex.the_arena_init_size: long = [system dependent]: This controls the main memory arena’s initial size in bytes. For CPU runs, the default is 0, whereas for GPU runs, the default is set at run time to 3/4 of the system’s device memory.

Tip

Since amrex v24.08, instead of amrex.the_arena_init_size=10000000000, one can use amrex.the_arena_init_size=10'000'000'000 or amrex.the_arena_init_size=1e10 to set ParmParse integer parameters like this one.

amrex.the_device_arena_init_size: long = 8388608 [8 MB]: This controls the GPU device arena’s initial size in bytes. For CPU runs, this is ignored. If the main arena uses the device memory (as opposed to managed memory), this parameter is also ignored.

amrex.the_managed_arena_init_size: long = 8388608 [8 MB]: This controls the managed device arena’s initial size in bytes. For CPU runs, this is ignored. If the main arena uses the managed memory (as opposed to device memory), this parameter is also ignored.

amrex.the_pinned_arena_init_size: long = [system dependent]: This controls the pinned host memory arena’s initial size in bytes. The default is 8 MB for CPU runs. For GPU runs it’s set to half of the GPU device memory by default.

amrex.the_comms_arena_init_size: long = 8388608 [8 MB]: This controls the MPI communication memory arena’s initial size in bytes.

amrex.the_arena_release_threshold: long = LONG_MAX: This controls the release threshold of the main arena.

amrex.the_device_arena_release_threshold: long = LONG_MAX: This controls the release threshold of the device arena.

amrex.the_managed_arena_release_threshold: long = LONG_MAX: This controls the release threshold of the managed arena.

amrex.the_pinned_arena_release_threshold: long = LONG_MAX: This controls the release threshold of the pinned arena.

amrex.the_comms_arena_release_threshold: long = LONG_MAX: This controls the release threshold of the communication arena.

amrex.the_async_arena_release_threshold: long = LONG_MAX: This controls the release threshold of the asynchronous arena. Note that this is only relevant for the CUDA (>= 11.2) and HIP backends that support stream-ordered memory allocator.

amrex.the_arena_defragmentation: bool = true: This controls the defragmentation behavior of the main arena.

amrex.the_device_arena_defragmentation: bool = true: This controls the defragmentation behavior of the device arena.

amrex.the_managed_arena_defragmentation: bool = true: This controls the defragmentation behavior of the managed arena.

amrex.the_pinned_arena_defragmentation: bool = true: This controls the defragmentation behavior of the pinned arena.

amrex.the_comms_arena_defragmentation: bool = [build dependent]: This controls the defragmentation behavior of the communication arena. The default is false for HIP builds and true for others.

amrex.the_arena_is_managed: bool = false: This controls if AMReX uses the managed memory for the main arena. This is only relevant for GPU runs.

amrex.abort_on_out_of_gpu_memory: bool = false: This controls if AMReX should simply abort when the reported free device memory is less than the amount an arena is asked to allocate. Note that for managed memory it’s possible to allocate more than the amount of free device memory available. However, the code will be very slow. This parameter is only relevant for GPU runs.

amrex.mf.alloc_single_chunk: bool = false: This controls if all the data in a FabArray (including MultiFab) are in a contiguous chunk of memory.

amrex.vector_growth_factor: amrex::Real = 1.5: This controls the growth factor of amrex::PODVector and its derived classes such as amrex::Gpu::DeviceVector, amrex::Gpu::ManagedVector, etc. A smaller value can avoid wasting memory, but it may result in a performance penalty during resizing.

Particles

particles.do_tiling: bool = false: This controls whether tiling is enabled for particle containers.

particles.tile_size: int array = 1024000 8 8: When tiling is enabled, this is the default tile size. Note that a big number like 1024000 effectively turns tiling off in that direction.

particles.do_mem_efficient_sort: bool = true: This parameter controls whether the more memory efficient method will be used for sorting particles.

particles.particles_nfiles: int = 256: This is the maximum number of binary files per level for a particle container when writing checkpoint and plot files for particles. The special value of -1 indicates one file per process.

Tiling

fabarray.mfiter_tile_size: int array = [build dependent]: This is the default size for tiling. For GPU runs, it is disabled by default. For CPU runs, it is disabled by default in 1D and 2D, but enabled in 3D with a tile size of 8 in the y and z-directions.

fabarray.comm_tile_size: int array = [build dependent]: This is the default tiling size used in moving data in and out of the MPI communication buffer . It is disabled by default for GPU runs, but enabled for CPU runs with a tile size of 8 in the y and z-directions (if they exist).

Time Integration

All these parameters are optional for constructing or configuring a TimeIntegrator object.

integration.type: string = [none]

This parameter sets the type of time integrator to use. The supported values are:

ForwardEuler
RungeKutta
SUNDIALS

integration.time_step: amrex::Real = [none]

This parameter sets the fixed time step size to use.

For SUNDIALS methods, this parameter sets a fixed step size for single rate methods (e.g., ERK) or a fixed slow time scale step size for multirate methods (e.g., EX-MRI).

Runge–Kutta Methods

These parameters are relevant only when integration.type is “RungeKutta”.

integration.rk.type: string = [none]

This parameter sets the Runge–Kutta method to use. The supported values are:

User
ForwardEuler
Trapezoid
SSPRK3
RK4

User-specificed Runge–Kutta Method

When integration.rk.type is “User”, the following parameters can be used to set a user-specificed explicit Butcher tableau,

\[\begin{split}B \; \equiv \; \begin{array}{r|c} c & A \\ \hline & b \\ & \tilde{b} \end{array},\end{split}\]

where, for a method with \(s\) stages, \(c\), and \(b\), \(\tilde{b}\) are arrays of \(s\) values and \(A\) is a lower triangular \(s \times s\) matrix.

integration.rk.nodes: amrex::Real array = [none]: The \(c\) values of the Butcher tableau.

integration.rk.tableau: amrex::Real array = [none]: The \(A\) values of the Butcher tableau.

integration.rk.weights: amrex::Real array = [none]: The \(b\) values of the Butcher tableau.

integration.rk.extended_weights: amrex::Real array = [none]: The \(\tilde{b}\) values of the Butcher tableau. These values are only required if the method has an embedding.

SUNDIALS

These parameters are relevant only when support for SUNDIALS time integrators is enabled (see Using SUNDIALS) and integration.type is “SUNDIALS”.

Methods

integration.sundials.type: string = ERK

This parameter sets type of SUNDIALS time integrator to use. See the table below for supported values.

Parameter Value	SUNDIALS Method Type
ERK	Explicit Runge-Kutta method
DIRK	Diagonally Implicit Runge-Kutta method
IMEX-RK	Implicit-Explicit Additive Runge-Kutta method
EX-MRI	Explicit Multirate Infinitesimal method
IM-MRI	Implicit Multirate Infinitesimal method
IMEX-MRI	Implicit-Explicit Multirate Infinitesimal method

integration.sundials.fast_type: string = ERK: When using a multirate method, this parameter sets the type of time integrator to use at the fast time scale. Currently, ERK and DIRK methods are supported.

integration.sundials.method: string = [none]

This parameter sets the name of the specific time integration method to use. See the sections listed below in the SUNDIALS documentation for valid method names. Note, IMEX method must be specified using the integration.sundials.method_i and integration.sundials.method_e parameters. If a method name is not provided, the SUNDIALS default method is used.

integration.sundials.method_i: string = [none]: When using an IMEX method, both the implicit and explicit methods must be specified. This parameter can be used to select the implicit portion of the IMEX method. See the ImEx methods section in the SUNDIALS documentation for valid method names. If a method name is not provided, the SUNDIALS default method is used.

integration.sundials.method_e: string = [none]: When using an IMEX method, both the implicit and explicit methods must be specified. This parameter can be used to select the explicit portion of the IMEX method. See the ImEx methods section in the SUNDIALS documentation for valid method names. If a method name is not provided, the SUNDIALS default method is used.

integration.sundials.fast_method: string = [none]: When using a multirate method, this parameter sets the method to use at the fast time scale. If a method name is not provided, the SUNDIALS default method is used.

Step Sizes

Note

The parameter integration.time_step is used to set a fixed step size with single rate methods (e.g., ERK) or a fixed slow time scale step size with multirate methods (e.g., EX-MRI).

integration.fast_time_step: amrex::Real = [none]: When using a multirate method, this parameter sets the fixed step size to use at the fast time scale.

integration.use_adaptive_time_step: bool = false: This parameter enables adaptive time step sizes with single rate methods (e.g., ERK) or adaptive time step sizes at the slow time scale with multirate methods (e.g., EX-MRI).

integration.use_adaptive_fast_time_step: bool = false: This parameter enables adaptive time step sizes at the fast time scale with multirate methods (e.g., EX-MRI).

integration.sundials.max_num_steps: int = 500: When using the evolve method, which allows the integrator to take multiple time steps to reach the specified output time, this parameter sets the maximum number of time steps allowed before reaching the output time. If this limit is reached before the output time, SUNDIALS will return an error.

integration.sundials.stop_time: amrex::Real = [none]: When using the evolve method (especially with adaptive time step sizes), the SUNDIALS integrator may step past the requested output time and return an interpolated solution at the requested time. This parameter will force the integrator to stop and return the solution at the requested time.

Tolerances

When using adaptive time step sizes or an implicit method (e.g., DIRK), selecting appropriate tolerances for the target application is a critical factor in method performance. For advice on selecting tolerances see the SUNDIALS documentation.

integration.rel_tol: amrex::Real = 1.e-4: Relative tolerance for temporal error control.

integration.abs_tol: amrex::Real = 1.e-9: Absolute tolerance for temporal error control.

integration.fast_rel_tol: amrex::Real = 1.e-4: Relative tolerance for the temporal error at the fast time scale with multrate methods.

integration.fast_abs_tol: amrex::Real = 1.e-9: Absolute tolerance for the temporal error at the fast time scale with multrate methods.

Algebraic Solvers

The parameters provide control over the nonlinear and linear solvers utilized with implicit methods (e.g., DIRK).

integration.sundials.nonlinear_solver: string = Newton

The nonlinear solver used with single rate methods (e.g., DIRK) or at the slow time scale with multirate methods (e.g., IM-MRI). The supported values are:

Newton
fixed-point

integration.sundials.max_nonlinear_iters: int = 3: The maximum number of nonlinear iterations allowed per solve with single rate methods (e.g., DIRK) or at the slow time scale with multirate methods (e.g., IM-MRI).

integration.sundials.linear_solver: string = GMRES: The linear solver used with Newton’s method for single rate methods (e.g., DIRK) or at the slow time scale with multirate methods (e.g., IM-MRI).

integration.sundials.max_linear_iters: int = 5: The maximum number of linear iterations allowed per solve with single rate methods (e.g., DIRK) or at the slow time scale with multirate methods (e.g., IM-MRI).

integration.sundials.fast_nonlinear_solver: string = Newton

The nonlinear solver used at the fast time scale with multirate methods (e.g., when the fast method is DIRK). The supported values are:

Newton
fixed-point

integration.sundials.fast_max_nonlinear_iters: int = 3: The maximum number of nonlinear iterations allowed per solve at the fast time scale (e.g., when the fast method is DIRK).

integration.sundials.fast_linear_solver: string = GMRES: The linear solver used with Newton’s method at the fast time scale with multirate methods (e.g., when the fast method is DIRK).

integration.sundials.fast_max_linear_iters: int = 5: The maximum number of linear iterations allowed at the fast time scale per solve with multirate methods (e.g., when the fast method is DIRK).

Tiny Profiler

These parameters are ignored unless profiling with TinyProfiler is enabled.

tiny_profiler.verbose: int = 0: If this value is greater than 0, messages about entering or leaving profiled regions will be printed on the I/O process.

tiny_profiler.print_threshold: double = 1.0: In the profiling report, regions with very small run times are not listed individually. Instead, they are included in a section named “Other”. This parameter specifies the maximum inclusive run time that the “Other” section can take in percent relative to the total run time.

tiny_profiler.device_synchronize_around_region: bool = false: This parameter is only relevant for GPU runs. If it is set to true, the current GPU stream is synchronized when entering and leaving a profiling region. Because GPU kernels are asynchronous, time measurements without synchronization could be misleading. Enabling this parameter can provide more accurate measurements. However, the added synchronization points, which are unnecessary for correctness, could potentially degrade the performance.

tiny_profiler.enabled: bool = true: New in version 24.09: Runtime parameter tiny_profiler.enabled`.

This parameter can be used to disable tiny profiling including CArena memory profiling at run time.

tiny_profiler.memprof_enabled: bool = true: New in version 24.09: Runtime parameter tiny_profiler.memprof_enabled.

This parameter can be used to disable CArena memory profiling at run time. If tiny_profiler.enabled is false, this parameter has no effects.

tiny_profiler.output_file: string = [empty]: New in version 24.09: Runtime parameter tiny_profiler.output_file.

If this parameter is empty, the output of tiny profiling is dumped on the default out stream of AMReX. If it’s not empty, it specifies the file name for the output. Note that /dev/null is a special name that means no output.