Block-Structured AMR Software Framework
Loading...
Searching...
No Matches
AMReX_CompositeSystemUtil.H File Reference
#include <AMReX_Config.H>
#include <AMReX_AlgMG.H>
#include <AMReX_GpuContainers.H>
#include <AMReX_HypreIJIface.H>
#include <AMReX_ParallelDescriptor.H>
#include <AMReX_ParallelReduce.H>
#include <AMReX_ParmParse.H>
#include <AMReX_Reduce.H>
#include <AMReX_Scan.H>
#include <AMReX_SpMatrix.H>
#include <AMReX_Vector.H>
#include <HYPRE_IJ_mv.h>
#include <algorithm>
#include <array>
#include <cmath>
#include <fstream>
#include <functional>
#include <iomanip>
#include <limits>
#include <map>
#include <sstream>
#include <string>
#include <utility>

Go to the source code of this file.

Classes

struct  amrex::composite::Triplet
 (row, column, value) of a matrix entry with global indices More...
struct  amrex::composite::EntryGather
 With setup reuse, the triplets of later assemblies are summed directly into a matrix's value array, each entry adding its triplets in increasing triplet order so that the sums do not depend on thread scheduling: the triplets of entry e are trip[offset[e]:offset[e+1]]. More...
struct  amrex::composite::SetupReuse
 The state of setup reuse (HypreMLABecLap::setReuseSolverSetup, HypreMLNodeLap::setReuseSolverSetup). More...
struct  amrex::composite::ReusedSolve
struct  amrex::composite::OwnedCSR
 Rows owned by this rank in CSR form, columns sorted within each row and duplicates summed. More...

Namespaces

namespace  amrex
namespace  amrex::composite

Functions

void amrex::composite::define_gather (EntryGather &g, Vector< Long > const &entry, Long nentries)
void amrex::composite::gather_sum (EntryGather const &g, Real const *vals, Real *out, Long nentries)
bool amrex::composite::ij_entry_slots (HYPRE_IJMatrix A, Gpu::DeviceVector< HYPRE_BigInt > const &rows, Gpu::DeviceVector< HYPRE_BigInt > const &cols, MPI_Comm comm, EntryGather &g)
 The entries of the assembled hypre IJ matrix A (its diag values first, then its offd values) that the triplets (rows, cols, all in this rank's rows) were summed into, as an EntryGather for in-place updates with ij_update_values. Collective over comm: returns false on all ranks (and g empty) if hypre dropped an entry on any rank.
void amrex::composite::ij_update_values (HYPRE_IJMatrix A, EntryGather const &g, Gpu::DeviceVector< Real > const &vals)
ReusedSolve amrex::composite::ij_solve_reused (HypreIJIface &ij, bool reused, bool throw_exception, Real reltol, Real abstol, int maxiter, MPI_Comm comm, int verbose, std::string const &prefix, std::function< bool(Real)> const &converged, std::function< void()> const &restore_x)
 Solve with ij. With a reused setup (reused: kept from an earlier matrix), a solve that blows up (a hypre error on any rank, or a non-finite residual) or does not converge (converged, of the final relative residual norm) is retried once with the setup redone, from the current solution or, after a blow-up, from the initial guess (restore_x reloads it); the caller's failure policy (throw_exception) applies to the retry. Collective over comm.
ReusedSolve amrex::composite::algmg_solve_kept (AlgMG< Real > &mg, SpMatrix< Real > &mat, bool kept, bool throw_exception, AlgVector< Real > &x, AlgVector< Real > const &b, Real bnorm, MPI_Comm comm, int verbose, std::string const &prefix, std::function< bool(Real)> const &converged, std::function< void()> const &restore_x)
 Solve with mg, whose tolerances and iteration limit are set. With a kept hierarchy (kept: from an earlier matrix), a solve that blows up or does not converge (converged, of the residual norm relative to bnorm) is retried once with the hierarchy rebuilt for mat, from the current solution or, after a blow-up, from the initial guess (restore_x reloads it); the caller's failure policy (throw_exception) applies to the retry. Collective over comm.
std::string amrex::composite::to_sci (double x)
 Scientific notation for messages (std::to_string prints 0.000000 for 1e-7).
Vector< Triplet > amrex::composite::exchange_triplets (Vector< Triplet > &&trips, Vector< Long > const &rank_offset, MPI_Comm comm)
 Send each triplet to the rank owning its row (rank p owns rows [rank_offset[p], rank_offset[p+1])) and return the triplets received by this rank, including its own.
template<typename I>
void amrex::composite::localize_triplets (Gpu::DeviceVector< I > &rows, Gpu::DeviceVector< I > &cols, Gpu::DeviceVector< Real > &vals, Long row_begin, Long nrows, Vector< Long > const &rank_offset, MPI_Comm comm)
 Send the (row, column, value) triplets in device memory whose rows are owned by other ranks to their owners, through the host (collective over comm). Afterwards the arrays hold the triplets of this rank's rows [row_begin, row_begin+nrows): its own in their order, then the received ones in rank order.
void amrex::composite::triplets_to_spmatrix (Gpu::DeviceVector< Long > &rows, Gpu::DeviceVector< Long > &cols, Gpu::DeviceVector< Real > &vals, Long row_begin, Long nrows, Vector< Long > const &rank_offset, MPI_Comm comm, SpMatrix< Real > &mat, bool localized=false)
 Build an SpMatrix of this rank's rows [row_begin, row_begin+nrows) from (row, column, value) triplets in device memory, without copying them to the host: the device counterpart of exchange_triplets, OwnedCSR::define and OwnedCSR::toSpMatrix.
std::map< std::string, int > & amrex::composite::dump_counters ()
 Dumps written so far per dump prefix, over all solver objects of the run.
bool amrex::composite::dump_pending (std::string const &prefix, int max)
 Whether a dump with this prefix is still pending (fewer than max written).
int amrex::composite::next_dump_index (std::string const &prefix, int max)
 Index of the next dump for this prefix, or -1 once max have been written.
template<typename T>
Vector< T > amrex::composite::gather_on_root (Vector< T > const &v, MPI_Comm comm)
 Gather the chunks of a distributed array on rank 0 (ranks in order); other ranks return an empty vector. The total must fit in int.
bool amrex::composite::dump_single_file (Long total, Long gather_max, MPI_Comm comm)
 Whether a dump of total values goes to one file written by rank 0 (always on one rank). Above gather_max, or above what MPI's int counts can address, every rank writes its own part to <file>.<rank>.
std::ofstream amrex::composite::open_dump (std::string const &name)
 Open a dump file for writing, aborting on failure.
void amrex::composite::check_dump (std::ofstream const &ofs, std::string const &name)
 Abort if writing a dump file failed.
void amrex::composite::write_row_map (std::string const &file, std::string const &title, Vector< std::array< Long, 4 > > const &info, Vector< Real > const &scaling, Long row_begin, MPI_Comm comm, Long gather_max)
 Write the row map of a composite system: one line per row with the 1-based row, AMR level, index (i j k) and the unknown's scaling. info holds (level,i,j,k) of this rank's rows, which start at the global row row_begin. Gathered on rank 0 or one file per rank, see dump_single_file.
void amrex::composite::algmg_parse_options (AlgMG< Real > &mg, std::string const &prefix, AlgMGKrylovSolver krylov)
 Configure an AlgMG solver from <prefix>.algmg.*: verbose, krylov (none|bicgstab|gmres|pcg; default krylov), smoother (jacobi|l1_jacobi|chebyshev|l1_gauss_seidel), interp_type (direct|mm_ext|mm_ext_i), bottom_solver (jacobi|bicgstab|gmres|direct), strong_threshold, max_levels, max_coarse_size, agg_num_levels, pmax_elmts, trunc_factor, pre_smooth, post_smooth, cheby_degree.
void amrex::composite::write_matrix_market (std::string const &file, OwnedCSR const &csr, Long nglobal, MPI_Comm comm, Long gather_max)
 Write a distributed matrix in Matrix Market coordinate format (1-based indices, "general"), gathered on rank 0 or one file per rank (see dump_single_file). A per-rank file holds this rank's rows with the global dimensions; concatenating the entry lines gives the matrix.
void amrex::composite::write_vector_market (std::string const &file, Vector< Real > const &v, Long nglobal, Long row_begin, MPI_Comm comm, Long gather_max)
 Write a distributed vector (this rank's rows start at the global row row_begin) as a Matrix Market dense array, gathered on rank 0 or one file per rank (see dump_single_file); a per-rank file gives its global row range in a comment.

Detailed Description

Helpers for the composite (all AMR levels) solvers such as HypreMLABecLap: an owned-row CSR built from (row, column, value) triplets, the in-place update of an assembled matrix from new triplets (setup reuse), the AlgMG backend options, and Matrix Market output of the assembled systems for analysis outside AMReX.