Grid Creation

To run an AMReX-based application you must specify the domain size by specifying n_cell – this is the number of cells spanning the domain in each coordinate direction at level 0.

Users often specify max_grid_size as well. The default load balancing algorithm then divides the domain in every direction so that each grid is no longer than max_grid_size in that direction. If not specified by the user, max_grid_size defaults to 128 in 2D and 32 in 3D (in each coordinate direction).

Another popular input is blocking_factor. The value of blocking_factor constrains grid creation in that in that each grid must be divisible by blocking_factor. Note that both the domain (at each level) and max_grid_size must be divisible by blocking_factor, and that blocking_factor must be either 1 or a power of 2 (otherwise the gridding algorithm would not in fact create grids divisible by blocking_factor because of how blocking_factor is used in the gridding algorithm).

If not specified by the user, blocking_factor defaults to 8 in each coordinate direction. The typical purpose of blocking_factor is to ensure that the grids will be sufficiently coarsenable for good multigrid performance.

There is one more default behavior to be aware of. There is a boolean refine_grid_layout that defaults to true but can be over-ridden at run-time. If refine_grid_layout is true and the number of grids created is less than the number of processors (Ngrids < Nprocs), then grids will be further subdivided until Ngrids >= Nprocs.

Caveat: if subdividing the grids to achieve Ngrids >= Nprocs would violate the blocking_factor criterion then additional grids are not created and the number of grids will remain less than the number of processors

Note that n_cell must be given as three separate integers, one for each coordinate direction.

However, max_grid_size and blocking_factor can be specified as a single value applying to all coordinate directions, or as separate values for each direction.

  • If max_grid_size (or blocking_factor) is specified as multiple integers then the first integer applies to level 0, the second to level 1, etc. If you don’t specify as many integers as there are levels, the final value will be used for the remaining levels.

  • If different values of max_grid_size (or blocking_factor) are wanted for each coordinate direction, then max_grid_size_x, max_grid_size_y and max_grid_size_z (or blocking_factor_x, blocking_factor_y and blocking_factor_z) must be used. If you don’t specify as many integers as there are levels, the final value will be used for the remaining levels.

Additional notes:

  • To create identical grids of a specific size, e.g. of length m in each direction, then set max_grid_size = m and blocking_factor = m.

  • Note that max_grid_size is just an upper bound; with n_cell = 48 and max_grid_size = 32, we will typically have one grid of length 32 and one of length 16.

The grid creation process at level 0 proceeds as follows (if not using the KD-tree approach):

  1. The domain is initially defined by a single grid of size n_cell.

  2. If n_cell is greater than max_grid_size then the grids are subdivided until each grid is no longer than max_grid_size cells on each side. The blocking_factor criterion (ie that the length of each side of each grid is divisible by blocking_factor in that direction) is satisfied during this process.

  3. Next, if refine_grid_layout = true and there are more processors than grids at this level, then the grids at this level are further divided until Ngrids >= Nprocs (unless doing so would violate the blocking_factor criterion).

The creation of grids at levels > 0 begins by tagging cells at the coarser level and follows the Berger-Rigoutsos clustering algorithm with the additional constraints of satisfying the blocking_factor and max_grid_size criteria. An additional parameter becomes relevant here: the “grid efficiency”, specified as amr.grid_eff in the inputs file. This threshold value, which defaults to 0.7 (or 70%), is used to ensure that grids do not contain too large a fraction of un-tagged cells. We note that the grid creation process attempts to satisfy the amr.grid_eff constraint but will not do so if it means violating the blocking_factor criterion.

Users often like to ensure that coarse/fine boundaries are not too close to tagged cells; the way to do this is to set amr.n_error_buf to a large integer value (the default is 1). This parameter is used to increase the number of tagged cells before the grids are defined; if cell “(i,j,k)” satisfies the tagging criteria, then, for example, if amr.n_error_buf is 3, all cells in the 7x7x7 box from lower corner “(i-3,j-3,k-3)” to “(i+3,j+3,k+3)” will be tagged.