Block-Structured AMR Software Framework
amrex::VisMF Class Reference

File I/O for FabArray<FArrayBox>. Wrapper class for reading/writing FabArray<FArrayBox> objects to disk in various "smart" ways. More...

#include <AMReX_VisMF.H>

Inheritance diagram for amrex::VisMF:
amrex::VisMFBuffer

Classes

struct  FabOnDisk
 A structure containing info regarding an on-disk FAB. More...
 
struct  FabReadLink
 This structure is used to store the read order for each FabArray file. More...
 
struct  Header
 An on-disk FabArray<FArrayBox> contains this info in a header file. More...
 
struct  PersistentIFStream
 This structure is used to store file ifstreams that remain open. More...
 

Public Types

enum  How { OneFilePerCPU , NFiles }
 How we write out FabArray<FArrayBox>s. These are deprecated, we always use NFiles. For OneFilePerCPU, set NFiles to NProcs. More...
 
- Public Types inherited from amrex::VisMFBuffer
enum  { IO_Buffer_Size = 262144 * 8 }
 We try to do I/O with buffers of this size. More...
 
using Setbuf_Char_Type = char
 The type of a char buffer required by [p]setbuf(). More...
 
using IO_Buffer = Vector< Setbuf_Char_Type >
 A simple character buffer for setbuf() usage. More...
 

Public Member Functions

 VisMF (std::string fafab_name)
 Construct by reading in the on-disk VisMF of the specified name. The FABs in the on-disk FabArray are read on demand unless the entire FabArray is requested. The name here is the name of the FabArray not the name of the on-disk files. More...
 
 ~VisMF ()=default
 
 VisMF (const VisMF &)=delete
 
 VisMF (VisMF &&)=delete
 
VisMFoperator= (const VisMF &)=delete
 
VisMFoperator= (VisMF &&)=delete
 
int nComp () const
 The number of components in the on-disk FabArray<FArrayBox>. More...
 
int nGrow () const
 The grow factor of the on-disk FabArray<FArrayBox>. More...
 
IntVect nGrowVect () const
 
int size () const
 
const BoxArrayboxArray () const
 The BoxArray of the on-disk FabArray<FArrayBox>. More...
 
Real min (int fabIndex, int nComp) const
 The min of the FAB (in valid region) at specified index and component. More...
 
Real min (int nComp) const
 The min of the FabArray (in valid region) at specified component. More...
 
Real max (int fabIndex, int nComp) const
 The max of the FAB (in valid region) at specified index and component. More...
 
Real max (int nComp) const
 The max of the FabArray (in valid region) at specified component. More...
 
const FArrayBoxGetFab (int fabIndex, int compIndex) const
 The FAB at the specified index and component. Reads it from disk if necessary. This reads only the specified component. More...
 
void clear (int fabIndex, int compIndex)
 Delete()s the FAB at the specified index and component. More...
 
void clear (int fabIndex)
 Delete()s the FAB at the specified index (all components). More...
 
void clear ()
 Delete()s all the FABs. More...
 
FArrayBoxreadFAB (int idx, const std::string &mf_name)
 Read the entire fab (all components). More...
 
FArrayBoxreadFAB (int idx, int icomp)
 Read the specified fab component. More...
 

Static Public Member Functions

static std::ifstream * OpenStream (const std::string &fileName)
 Open the stream if it is not already open Close the stream if not persistent or forced Close all open streams. More...
 
static void CloseStream (const std::string &fileName, bool forceClose=false)
 
static void DeleteStream (const std::string &fileName)
 
static void CloseAllStreams ()
 
static bool NoFabHeader (const VisMF::Header &hdr)
 
static Long Write (const FabArray< FArrayBox > &mf, const std::string &name, VisMF::How how=NFiles, bool set_ghost=false)
 Write a FabArray<FArrayBox> to disk in a "smart" way. Returns the total number of bytes written on this processor. If set_ghost is true, sets the ghost cells in the FabArray<FArrayBox> to one-half the average of the min and max over the valid region of each contained FAB. More...
 
static void AsyncWrite (const FabArray< FArrayBox > &mf, const std::string &mf_name, bool valid_cells_only=false)
 
static void AsyncWrite (FabArray< FArrayBox > &&mf, const std::string &mf_name, bool valid_cells_only=false)
 
static Long WriteOnlyHeader (const FabArray< FArrayBox > &mf, const std::string &mf_name, VisMF::How how=NFiles)
 Write only the header-file corresponding to FabArray<FArrayBox> to disk without the corresponding FAB data. This writes BoxArray information (which might still be needed by data post-processing tools such as yt) when the FAB data is not needed. Returns the total number of bytes written on this processor. More...
 
static void RemoveFiles (const std::string &name, bool verbose=false)
 this will remove nfiles associated with name and the header More...
 
static void Read (FabArray< FArrayBox > &mf, const std::string &name, const char *faHeader=nullptr, int coordinatorProc=ParallelDescriptor::IOProcessorNumber(), int allow_empty_mf=0)
 Read a FabArray<FArrayBox> from disk written using VisMF::Write(). If the FabArray<FArrayBox> fafab has been fully defined, the BoxArray on the disk must match the BoxArray in fafab. If it is constructed with the default constructor, the BoxArray on the disk will be used and a new DistributionMapping will be made. A pre-read FabArray header can be passed in to avoid a read and broadcast. More...
 
static bool Exist (const std::string &name)
 Does FabArray exist? More...
 
static void ReadFAHeader (const std::string &fafabName, Vector< char > &header)
 Read only the header of a FabArray, header will be resized here. More...
 
static bool Check (const std::string &name)
 Check if the multifab is ok, false is returned if not ok. More...
 
static Long FileOffset (std::ostream &os)
 The file offset of the passed ostream. More...
 
static int GetNOutFiles ()
 
static void SetNOutFiles (int noutfiles, MPI_Comm comm=ParallelDescriptor::Communicator())
 
static int GetMFFileInStreams ()
 
static void SetMFFileInStreams (int nstreams, MPI_Comm comm=ParallelDescriptor::Communicator())
 
static int GetVerbose ()
 
static void SetVerbose (int v)
 
static VisMF::Header::Version GetHeaderVersion ()
 
static void SetHeaderVersion (VisMF::Header::Version version)
 
static bool GetGroupSets ()
 
static void SetGroupSets (bool groupsets)
 
static bool GetSetBuf ()
 
static void SetSetBuf (bool setbuf)
 
static bool GetUseSingleRead ()
 
static void SetUseSingleRead (bool usesingleread)
 
static bool GetUseSingleWrite ()
 
static void SetUseSingleWrite (bool usesinglewrite)
 
static bool GetCheckFilePositions ()
 
static void SetCheckFilePositions (bool cfp)
 
static bool GetUsePersistentIFStreams ()
 
static void SetUsePersistentIFStreams (bool usepifs)
 
static bool GetUseSynchronousReads ()
 
static void SetUseSynchronousReads (bool usepsr)
 
static bool GetUseDynamicSetSelection ()
 
static void SetUseDynamicSetSelection (bool usedss)
 
static std::string DirName (const std::string &filename)
 
static std::string BaseName (const std::string &filename)
 
static void Initialize ()
 
static void Finalize ()
 
- Static Public Member Functions inherited from amrex::VisMFBuffer
static Long GetIOBufferSize ()
 
static void SetIOBufferSize (Long iobuffersize)
 
static void ClearBuffer (IO_Buffer &buf)
 

Static Private Member Functions

static FabOnDisk Write (const FArrayBox &fab, const std::string &filename, std::ostream &os, Long &bytes)
 
static Long WriteHeaderDoit (const std::string &mf_name, VisMF::Header const &hdr)
 
static Long WriteHeader (const std::string &mf_name, VisMF::Header &hdr, int procToWrite=ParallelDescriptor::IOProcessorNumber(), MPI_Comm comm=ParallelDescriptor::Communicator())
 
static void FindOffsets (const FabArray< FArrayBox > &mf, const std::string &filePrefix, VisMF::Header &hdr, VisMF::Header::Version whichVersion, NFilesIter &nfi, MPI_Comm comm=ParallelDescriptor::Communicator())
 fileNumbers must be passed in for dynamic set selection [proc] More...
 
static FArrayBoxreadFAB (int idx, const std::string &mf_name, const Header &hdr, int whichComp=-1)
 Make a new FAB from a fab in a FabArray<FArrayBox> on disk. The returned *FAB will have either one component filled from fafab[fabIndex][whichComp] or fafab[fabIndex].nComp() components. whichComp == -1 means reads the whole FAB. Otherwise read just that component. More...
 
static void readFAB (FabArray< FArrayBox > &mf, int idx, const std::string &mf_name, const Header &hdr)
 Read the whole FAB into fafab[fabIndex]. More...
 
static void AsyncWriteDoit (const FabArray< FArrayBox > &mf, const std::string &mf_name, bool is_rvalue, bool valid_cells_only)
 

Private Attributes

std::string m_fafabname
 Name of the FabArray<FArrayBox>. More...
 
Header m_hdr
 The VisMF header as read from disk. More...
 
Vector< Vector< FArrayBox * > > m_pa
 We manage the FABs individually. More...
 

Static Private Attributes

static AMREX_EXPORT std::map< std::string, VisMF::PersistentIFStreampersistentIFStreams
 Persistent streams. These open on demand and should be closed when not needed with CloseAllStreams. ~VisMF also closes them. [filename, pifs]. More...
 
static AMREX_EXPORT int nOutFiles
 The number of files to write for a FabArray<FArrayBox>. More...
 
static AMREX_EXPORT int nMFFileInStreams
 
static AMREX_EXPORT int verbose
 
static AMREX_EXPORT VisMF::Header::Version currentVersion
 
static AMREX_EXPORT bool groupSets
 
static AMREX_EXPORT bool setBuf
 
static AMREX_EXPORT bool useSingleRead
 
static AMREX_EXPORT bool useSingleWrite
 
static AMREX_EXPORT bool checkFilePositions
 
static AMREX_EXPORT bool usePersistentIFStreams
 
static AMREX_EXPORT bool useSynchronousReads
 
static AMREX_EXPORT bool useDynamicSetSelection
 
static AMREX_EXPORT bool allowSparseWrites
 

Additional Inherited Members

- Static Protected Attributes inherited from amrex::VisMFBuffer
static AMREX_EXPORT Long ioBufferSize
 -— the settable buffer size More...
 

Detailed Description

File I/O for FabArray<FArrayBox>. Wrapper class for reading/writing FabArray<FArrayBox> objects to disk in various "smart" ways.

Member Enumeration Documentation

◆ How

How we write out FabArray<FArrayBox>s. These are deprecated, we always use NFiles. For OneFilePerCPU, set NFiles to NProcs.

Enumerator
OneFilePerCPU 
NFiles 

Constructor & Destructor Documentation

◆ VisMF() [1/3]

amrex::VisMF::VisMF ( std::string  fafab_name)
explicit

Construct by reading in the on-disk VisMF of the specified name. The FABs in the on-disk FabArray are read on demand unless the entire FabArray is requested. The name here is the name of the FabArray not the name of the on-disk files.

◆ ~VisMF()

amrex::VisMF::~VisMF ( )
default

◆ VisMF() [2/3]

amrex::VisMF::VisMF ( const VisMF )
delete

◆ VisMF() [3/3]

amrex::VisMF::VisMF ( VisMF &&  )
delete

Member Function Documentation

◆ AsyncWrite() [1/2]

void amrex::VisMF::AsyncWrite ( const FabArray< FArrayBox > &  mf,
const std::string &  mf_name,
bool  valid_cells_only = false 
)
static

◆ AsyncWrite() [2/2]

void amrex::VisMF::AsyncWrite ( FabArray< FArrayBox > &&  mf,
const std::string &  mf_name,
bool  valid_cells_only = false 
)
static

◆ AsyncWriteDoit()

void amrex::VisMF::AsyncWriteDoit ( const FabArray< FArrayBox > &  mf,
const std::string &  mf_name,
bool  is_rvalue,
bool  valid_cells_only 
)
staticprivate

◆ BaseName()

std::string amrex::VisMF::BaseName ( const std::string &  filename)
static

◆ boxArray()

const BoxArray & amrex::VisMF::boxArray ( ) const

The BoxArray of the on-disk FabArray<FArrayBox>.

◆ Check()

bool amrex::VisMF::Check ( const std::string &  name)
static

Check if the multifab is ok, false is returned if not ok.

◆ clear() [1/3]

void amrex::VisMF::clear ( )

Delete()s all the FABs.

◆ clear() [2/3]

void amrex::VisMF::clear ( int  fabIndex)

Delete()s the FAB at the specified index (all components).

◆ clear() [3/3]

void amrex::VisMF::clear ( int  fabIndex,
int  compIndex 
)

Delete()s the FAB at the specified index and component.

◆ CloseAllStreams()

void amrex::VisMF::CloseAllStreams ( )
static

◆ CloseStream()

void amrex::VisMF::CloseStream ( const std::string &  fileName,
bool  forceClose = false 
)
static

◆ DeleteStream()

void amrex::VisMF::DeleteStream ( const std::string &  fileName)
static

◆ DirName()

std::string amrex::VisMF::DirName ( const std::string &  filename)
static

◆ Exist()

bool amrex::VisMF::Exist ( const std::string &  name)
static

Does FabArray exist?

◆ FileOffset()

Long amrex::VisMF::FileOffset ( std::ostream &  os)
static

The file offset of the passed ostream.

◆ Finalize()

void amrex::VisMF::Finalize ( )
static

◆ FindOffsets()

void amrex::VisMF::FindOffsets ( const FabArray< FArrayBox > &  mf,
const std::string &  filePrefix,
VisMF::Header hdr,
VisMF::Header::Version  whichVersion,
NFilesIter nfi,
MPI_Comm  comm = ParallelDescriptor::Communicator() 
)
staticprivate

fileNumbers must be passed in for dynamic set selection [proc]

◆ GetCheckFilePositions()

static bool amrex::VisMF::GetCheckFilePositions ( )
inlinestatic

◆ GetFab()

const FArrayBox & amrex::VisMF::GetFab ( int  fabIndex,
int  compIndex 
) const

The FAB at the specified index and component. Reads it from disk if necessary. This reads only the specified component.

◆ GetGroupSets()

static bool amrex::VisMF::GetGroupSets ( )
inlinestatic

◆ GetHeaderVersion()

static VisMF::Header::Version amrex::VisMF::GetHeaderVersion ( )
inlinestatic

◆ GetMFFileInStreams()

static int amrex::VisMF::GetMFFileInStreams ( )
inlinestatic

◆ GetNOutFiles()

int amrex::VisMF::GetNOutFiles ( )
static

◆ GetSetBuf()

static bool amrex::VisMF::GetSetBuf ( )
inlinestatic

◆ GetUseDynamicSetSelection()

static bool amrex::VisMF::GetUseDynamicSetSelection ( )
inlinestatic

◆ GetUsePersistentIFStreams()

static bool amrex::VisMF::GetUsePersistentIFStreams ( )
inlinestatic

◆ GetUseSingleRead()

static bool amrex::VisMF::GetUseSingleRead ( )
inlinestatic

◆ GetUseSingleWrite()

static bool amrex::VisMF::GetUseSingleWrite ( )
inlinestatic

◆ GetUseSynchronousReads()

static bool amrex::VisMF::GetUseSynchronousReads ( )
inlinestatic

◆ GetVerbose()

static int amrex::VisMF::GetVerbose ( )
inlinestatic

◆ Initialize()

void amrex::VisMF::Initialize ( )
static

◆ max() [1/2]

Real amrex::VisMF::max ( int  fabIndex,
int  nComp 
) const

The max of the FAB (in valid region) at specified index and component.

◆ max() [2/2]

Real amrex::VisMF::max ( int  nComp) const

The max of the FabArray (in valid region) at specified component.

◆ min() [1/2]

Real amrex::VisMF::min ( int  fabIndex,
int  nComp 
) const

The min of the FAB (in valid region) at specified index and component.

◆ min() [2/2]

Real amrex::VisMF::min ( int  nComp) const

The min of the FabArray (in valid region) at specified component.

◆ nComp()

int amrex::VisMF::nComp ( ) const

The number of components in the on-disk FabArray<FArrayBox>.

◆ nGrow()

int amrex::VisMF::nGrow ( ) const

The grow factor of the on-disk FabArray<FArrayBox>.

◆ nGrowVect()

IntVect amrex::VisMF::nGrowVect ( ) const

◆ NoFabHeader()

bool amrex::VisMF::NoFabHeader ( const VisMF::Header hdr)
static

◆ OpenStream()

std::ifstream * amrex::VisMF::OpenStream ( const std::string &  fileName)
static

Open the stream if it is not already open Close the stream if not persistent or forced Close all open streams.

◆ operator=() [1/2]

VisMF& amrex::VisMF::operator= ( const VisMF )
delete

◆ operator=() [2/2]

VisMF& amrex::VisMF::operator= ( VisMF &&  )
delete

◆ Read()

void amrex::VisMF::Read ( FabArray< FArrayBox > &  mf,
const std::string &  name,
const char *  faHeader = nullptr,
int  coordinatorProc = ParallelDescriptor::IOProcessorNumber(),
int  allow_empty_mf = 0 
)
static

Read a FabArray<FArrayBox> from disk written using VisMF::Write(). If the FabArray<FArrayBox> fafab has been fully defined, the BoxArray on the disk must match the BoxArray in fafab. If it is constructed with the default constructor, the BoxArray on the disk will be used and a new DistributionMapping will be made. A pre-read FabArray header can be passed in to avoid a read and broadcast.

◆ readFAB() [1/4]

void amrex::VisMF::readFAB ( FabArray< FArrayBox > &  mf,
int  idx,
const std::string &  mf_name,
const Header hdr 
)
staticprivate

Read the whole FAB into fafab[fabIndex].

◆ readFAB() [2/4]

FArrayBox * amrex::VisMF::readFAB ( int  idx,
const std::string &  mf_name 
)

Read the entire fab (all components).

◆ readFAB() [3/4]

FArrayBox * amrex::VisMF::readFAB ( int  idx,
const std::string &  mf_name,
const Header hdr,
int  whichComp = -1 
)
staticprivate

Make a new FAB from a fab in a FabArray<FArrayBox> on disk. The returned *FAB will have either one component filled from fafab[fabIndex][whichComp] or fafab[fabIndex].nComp() components. whichComp == -1 means reads the whole FAB. Otherwise read just that component.

◆ readFAB() [4/4]

FArrayBox * amrex::VisMF::readFAB ( int  idx,
int  icomp 
)

Read the specified fab component.

◆ ReadFAHeader()

void amrex::VisMF::ReadFAHeader ( const std::string &  fafabName,
Vector< char > &  header 
)
static

Read only the header of a FabArray, header will be resized here.

◆ RemoveFiles()

void amrex::VisMF::RemoveFiles ( const std::string &  name,
bool  verbose = false 
)
static

this will remove nfiles associated with name and the header

◆ SetCheckFilePositions()

static void amrex::VisMF::SetCheckFilePositions ( bool  cfp)
inlinestatic

◆ SetGroupSets()

static void amrex::VisMF::SetGroupSets ( bool  groupsets)
inlinestatic

◆ SetHeaderVersion()

static void amrex::VisMF::SetHeaderVersion ( VisMF::Header::Version  version)
inlinestatic

◆ SetMFFileInStreams()

void amrex::VisMF::SetMFFileInStreams ( int  nstreams,
MPI_Comm  comm = ParallelDescriptor::Communicator() 
)
static

◆ SetNOutFiles()

void amrex::VisMF::SetNOutFiles ( int  noutfiles,
MPI_Comm  comm = ParallelDescriptor::Communicator() 
)
static

◆ SetSetBuf()

static void amrex::VisMF::SetSetBuf ( bool  setbuf)
inlinestatic

◆ SetUseDynamicSetSelection()

static void amrex::VisMF::SetUseDynamicSetSelection ( bool  usedss)
inlinestatic

◆ SetUsePersistentIFStreams()

static void amrex::VisMF::SetUsePersistentIFStreams ( bool  usepifs)
inlinestatic

◆ SetUseSingleRead()

static void amrex::VisMF::SetUseSingleRead ( bool  usesingleread)
inlinestatic

◆ SetUseSingleWrite()

static void amrex::VisMF::SetUseSingleWrite ( bool  usesinglewrite)
inlinestatic

◆ SetUseSynchronousReads()

static void amrex::VisMF::SetUseSynchronousReads ( bool  usepsr)
inlinestatic

◆ SetVerbose()

static void amrex::VisMF::SetVerbose ( int  v)
inlinestatic

◆ size()

int amrex::VisMF::size ( ) const

of FABs in the VisMF. Equal to # of Boxes in the BoxArray.

◆ Write() [1/2]

Long amrex::VisMF::Write ( const FabArray< FArrayBox > &  mf,
const std::string &  name,
VisMF::How  how = NFiles,
bool  set_ghost = false 
)
static

Write a FabArray<FArrayBox> to disk in a "smart" way. Returns the total number of bytes written on this processor. If set_ghost is true, sets the ghost cells in the FabArray<FArrayBox> to one-half the average of the min and max over the valid region of each contained FAB.

◆ Write() [2/2]

VisMF::FabOnDisk amrex::VisMF::Write ( const FArrayBox fab,
const std::string &  filename,
std::ostream &  os,
Long &  bytes 
)
staticprivate

◆ WriteHeader()

Long amrex::VisMF::WriteHeader ( const std::string &  mf_name,
VisMF::Header hdr,
int  procToWrite = ParallelDescriptor::IOProcessorNumber(),
MPI_Comm  comm = ParallelDescriptor::Communicator() 
)
staticprivate

◆ WriteHeaderDoit()

Long amrex::VisMF::WriteHeaderDoit ( const std::string &  mf_name,
VisMF::Header const &  hdr 
)
staticprivate

◆ WriteOnlyHeader()

Long amrex::VisMF::WriteOnlyHeader ( const FabArray< FArrayBox > &  mf,
const std::string &  mf_name,
VisMF::How  how = NFiles 
)
static

Write only the header-file corresponding to FabArray<FArrayBox> to disk without the corresponding FAB data. This writes BoxArray information (which might still be needed by data post-processing tools such as yt) when the FAB data is not needed. Returns the total number of bytes written on this processor.

Member Data Documentation

◆ allowSparseWrites

bool amrex::VisMF::allowSparseWrites
staticprivate

◆ checkFilePositions

bool amrex::VisMF::checkFilePositions
staticprivate

◆ currentVersion

VisMF::Header::Version amrex::VisMF::currentVersion
staticprivate

◆ groupSets

bool amrex::VisMF::groupSets
staticprivate

◆ m_fafabname

std::string amrex::VisMF::m_fafabname
private

Name of the FabArray<FArrayBox>.

◆ m_hdr

Header amrex::VisMF::m_hdr
private

The VisMF header as read from disk.

◆ m_pa

Vector< Vector<FArrayBox*> > amrex::VisMF::m_pa
mutableprivate

We manage the FABs individually.

◆ nMFFileInStreams

int amrex::VisMF::nMFFileInStreams
staticprivate

◆ nOutFiles

int amrex::VisMF::nOutFiles
staticprivate

The number of files to write for a FabArray<FArrayBox>.

◆ persistentIFStreams

std::map< std::string, VisMF::PersistentIFStream > amrex::VisMF::persistentIFStreams
staticprivate

Persistent streams. These open on demand and should be closed when not needed with CloseAllStreams. ~VisMF also closes them. [filename, pifs].

◆ setBuf

bool amrex::VisMF::setBuf
staticprivate

◆ useDynamicSetSelection

bool amrex::VisMF::useDynamicSetSelection
staticprivate

◆ usePersistentIFStreams

bool amrex::VisMF::usePersistentIFStreams
staticprivate

◆ useSingleRead

bool amrex::VisMF::useSingleRead
staticprivate

◆ useSingleWrite

bool amrex::VisMF::useSingleWrite
staticprivate

◆ useSynchronousReads

bool amrex::VisMF::useSynchronousReads
staticprivate

◆ verbose

int amrex::VisMF::verbose
staticprivate

The documentation for this class was generated from the following files: