AMReX+MFEM

The Physical Problem

Consider a drop of dye with concentration $\phi$, passively advected by a fluid with prescribed motion. In each case shown below, $\phi$ is initialized using a cosine profile $\phi = 0.5(1 + \cos(\pi r/w))$ for $r < w$ and $\phi = 0$ otherwise, where $r$ is the distance from the blob center and $w = 0.2$ is the blob radius.

The Solution Procedure

AMReX solves for $\phi(\mathbf{x},t)$ using an explicit finite volume approach on a block-structured mesh that maintains conservation of $\phi$ and is first-order accurate in time and second-order accurate in space.

MFEM solves for $\phi(\mathbf{x},t)$ using a discontinuous piecewise-cubic finite element approach on an unstructured mesh of quadrilateral elements with explicit third-order time integration.

Each simulation uses multiple MPI processes and domain decomposition such that MPI processes are split between AMReX and MFEM with data exchange occurring every time step at user-defined coupling regions. The frameworks communicate field values and coordinates through MPI point-to-point messaging.

ParaView Visualization Capabilities

ParaView supports both AMReX native plotfiles and MFEM-generated ParaView-ready output. For coupled simulations, both outputs must be read separately but can be viewed together. The reading can be facilitated through pvsm or python state scripts. These tools enable automated animations in multiple formats (AVI, MP4, GIF) using pvpython with various rendering backends.

As part of the visualization capabilities in this work, we provide both Python state files and PVSM ParaView state files that can be used to quickly reproduce the plots and animations shown on this website. These state files can also be incorporated into debugging or analysis workflows to facilitate consistent visualization setups across different simulation runs and parameter studies. Additionally, we provide a Python state file demonstrating multi-layout visualization capabilities for coupled MFEM-AMReX simulations. The script creates three side-by-side render views: a complete surface representation using unified Viridis colormap, contour analysis with multiple isosurface levels rendered as wireframe overlays, and threshold analysis using layered opacity effects. All views are synchronized for animation playback with consistent color mapping and camera positioning. For guidance on using state files with ParaView, see the saving and loading state files subheading in the AMReX documentation.

Visualization of AMReX results is facilitated by the write_series_file_timestamp.sh script, which generates JSON-formatted .series files indexing plotfiles with timestamps. This follows AMReX documentation guidelines for time series files that can be efficiently loaded into ParaView or VisIt. The script can be re-executed as new plotfiles are generated, and ParaView's "Reload Files" functionality enables seamless incorporation of new timesteps during ongoing simulations.

This 2D example demonstrates the AMReX-MFEM coupling for scalar advection on a rectangular domain.

For this domain, we define a simple shearing flow where the flow moves left to right in the lower half and right to left in the upper half (i.e. $u = ±1$ and $v = 0$). The solution (aka the value of $\phi$) is computed by MFEM in the left half of the domain and by AMReX in the right half of the domain. The domain is periodic in both the $x$- and $y-$ directions.

This 3D example extends the scalar advection demonstration to three dimensions, The same MPI domain decomposition approach is used, with processes split between AMReX and MFEM frameworks (again with MFEM on the left).

The velocity field is analogous to that in the 2D case, with $u = ±1$ depending on the y-coordinate and $v = w = 0$. As above, the solution is computed by MFEM in the left half of the domain and by AMReX in the right half of the domain.

This example demonstrates two additional features: first, the ability to have multiple AMReX subdomains and multiple MFEM subdomains with multiple planes of intersection in a single simulation, and second, the ability of MFEM to naturally support a non-rectangular geometry.

In this problem the flow is counter-clockwise and is parallel to the domain boundaries as they appear in the animation. The flow in the semicircular regions is that of solid-body rotation. The racetrack example showcases the potential flexibility of the coupling framework for complex multiphysics problems requiring both structured and unstructured discretization approaches.

This example demonstrates advanced three-dimensional capabilities: the ability to have multiple AMReX subdomains and multiple MFEM subdomains with multiple planes of intersection in a single simulation, and the ability of MFEM to naturally support complex non-rectangular geometries with varying vertical dimensions.

The helix geometry features a variable height profile, with sections maintaining different elevations that create a banked track configuration. This showcases how the coupling framework can handle geometries with significant topographical variation.

In this problem the flow follows the helical domain boundaries as they appear in the animation, with vertical flow components accommodating the height transitions. The flow in the curved regions exhibits complex three-dimensional patterns due to the elevation changes.

The three-dimensional helix example showcases the enhanced flexibility of the coupling framework for complex multiphysics problems requiring both structured and unstructured discretization approaches across geometries with significant topographical variation.