Detailed Description

Wrapper class for PETSc matrix routines for matrix.

Definition at line 31 of file PETScMatrix.h.

#include <PETScMatrix.h>

Public Types
using	IndexType = PetscInt

Public Member Functions
	PETScMatrix ()

	PETScMatrix (const PetscInt nrows, const PETScMatrixOption &mat_op=PETScMatrixOption())
	Constructor for a square matrix partitioning with more options.

	PETScMatrix (const PetscInt nrows, const PetscInt ncols, const PETScMatrixOption &mat_op=PETScMatrixOption())
	Constructor for a rectangular matrix partitioning with more options.

	~PETScMatrix ()

	PETScMatrix (PETScMatrix const &A)

PETScMatrix &	operator= (PETScMatrix const &A)

void	finalizeAssembly (const MatAssemblyType asm_type=MAT_FINAL_ASSEMBLY)
	Perform MPI collection of assembled entries in buffer.

PetscInt	getNumberOfRows () const
	Get the number of rows.

PetscInt	getNumberOfColumns () const
	Get the number of columns.

PetscInt	getNumberOfLocalRows () const
	Get the number of local rows.

PetscInt	getNumberOfLocalColumns () const
	Get the number of local columns.

PetscInt	getRangeBegin () const
	Get the start global index of the rows of the same rank.

PetscInt	getRangeEnd () const
	Get the end global index of the rows in the same rank.

Mat &	getRawMatrix ()
	Get matrix reference.

Mat const &	getRawMatrix () const

void	setZero ()
	Set all entries to zero.

void	setRowsColumnsZero (std::vector< PetscInt > const &row_pos)
	Set the specified rows to zero except diagonal entries, i.e. \(A(k, j) = \begin{cases} 0.0, &j\not=k, j=1,2,\dots,k-1, k+1, \dots, n \\ 1.0, &j = k \end{cases}\), where \(k \in \mbox{row\_pos}\) This function must be called by all ranks.

void	set (const PetscInt i, const PetscInt j, const PetscScalar value)
	Set a single entry with a value.

void	add (const PetscInt i, const PetscInt j, const PetscScalar value)
	Add value to a single entry.

template<class T_DENSE_MATRIX >
void	add (RowColumnIndices< PetscInt > const &indices, const T_DENSE_MATRIX &sub_matrix)
	Add sub-matrix at positions given by global `indices`, in which negative index indicates ghost entry.

template<class T_DENSE_MATRIX >
void	add (std::vector< PetscInt > const &row_pos, std::vector< PetscInt > const &col_pos, const T_DENSE_MATRIX &sub_mat)
	Add a dense sub-matrix to a PETSc matrix.

void	viewer (const std::string &file_name, const PetscViewerFormat vw_format=PETSC_VIEWER_ASCII_MATLAB)

Private Member Functions
void	destroy ()

void	create (const PetscInt d_nz, const PetscInt o_nz)
	Create the matrix, configure memory allocation and set the related member data.

Private Attributes
Mat	A_ = nullptr
	PETSc matrix.

PetscInt	nrows_
	Number of the global rows.

PetscInt	ncols_
	Number of the global columns.

PetscInt	n_loc_rows_
	Number of the local rows.

PetscInt	n_loc_cols_
	Number of the local columns.

PetscInt	start_rank_
	Starting index in a rank.

PetscInt	end_rank_
	Ending index in a rank.

Friends
bool	finalizeMatrixAssembly (PETScMatrix &mat, const MatAssemblyType asm_type=MAT_FINAL_ASSEMBLY)
	General interface for the matrix assembly.

Member Typedef Documentation

◆ IndexType

using MathLib::PETScMatrix::IndexType = PetscInt

Definition at line 34 of file PETScMatrix.h.

Constructor & Destructor Documentation

◆ PETScMatrix() [1/4]

MathLib::PETScMatrix::PETScMatrix ( )

inline

Definition at line 37 of file PETScMatrix.h.

37{}

◆ PETScMatrix() [2/4]

MathLib::PETScMatrix::PETScMatrix	(	const PetscInt	nrows,
		const PETScMatrixOption &	mat_op = PETScMatrixOption() )

Constructor for a square matrix partitioning with more options.

Parameters

nrows	The number of rows of the matrix or the local matrix.
mat_op	The configuration information for creating a matrix.

Definition at line 23 of file PETScMatrix.cpp.

    : nrows_(nrows),
      ncols_(nrows),
      n_loc_rows_(PETSC_DECIDE),
      n_loc_cols_(mat_opt.n_local_cols)
{
    if (!mat_opt.is_global_size)
    {
        n_loc_rows_ = nrows;
        n_loc_cols_ = nrows;
        nrows_ = PETSC_DECIDE;
        ncols_ = PETSC_DECIDE;
    }
 
    create(mat_opt.d_nz, mat_opt.o_nz);
}

References create(), MathLib::PETScMatrixOption::d_nz, MathLib::PETScMatrixOption::is_global_size, and MathLib::PETScMatrixOption::o_nz.

◆ PETScMatrix() [3/4]

MathLib::PETScMatrix::PETScMatrix	(	const PetscInt	nrows,
		const PetscInt	ncols,
		const PETScMatrixOption &	mat_op = PETScMatrixOption() )

Constructor for a rectangular matrix partitioning with more options.

Parameters

nrows	The number of global or local rows.
ncols	The number of global or local columns.
mat_op	The configuration information for creating a matrix.

Definition at line 40 of file PETScMatrix.cpp.

    : nrows_(nrows),
      ncols_(ncols),
      n_loc_rows_(PETSC_DECIDE),
      n_loc_cols_(mat_opt.n_local_cols)
{
    if (!mat_opt.is_global_size)
    {
        nrows_ = PETSC_DECIDE;
        ncols_ = PETSC_DECIDE;
        n_loc_rows_ = nrows;
        n_loc_cols_ = ncols;
    }
 
    create(mat_opt.d_nz, mat_opt.o_nz);
}

References create(), MathLib::PETScMatrixOption::d_nz, MathLib::PETScMatrixOption::is_global_size, and MathLib::PETScMatrixOption::o_nz.

◆ ~PETScMatrix()

MathLib::PETScMatrix::~PETScMatrix ( )

inline

Definition at line 57 of file PETScMatrix.h.

57{ destroy(); }

MathLib::PETScMatrix::destroy

void destroy()

Definition PETScMatrix.h:217

References destroy().

◆ PETScMatrix() [4/4]

MathLib::PETScMatrix::PETScMatrix ( PETScMatrix const & A )

Definition at line 58 of file PETScMatrix.cpp.

    : nrows_(A.nrows_),
      ncols_(A.ncols_),
      n_loc_rows_(A.n_loc_rows_),
      n_loc_cols_(A.n_loc_cols_),
      start_rank_(A.start_rank_),
      end_rank_(A.end_rank_)
{
    MatConvert(A.A_, MATSAME, MAT_INITIAL_MATRIX, &A_);
}

References A_.

Member Function Documentation

◆ add() [1/3]

void MathLib::PETScMatrix::add	(	const PetscInt	i,
		const PetscInt	j,
		const PetscScalar	value )

inline

Add value to a single entry.

Parameters

i	The row index.
j	The column index.
value	The entry value.

Definition at line 124 of file PETScMatrix.h.

    {
        MatSetValue(A_, i, j, value, ADD_VALUES);
    }

References A_.

Referenced by add(), MathLib::addToMatrix(), and MathLib::setMatrix().

◆ add() [2/3]

template<class T_DENSE_MATRIX >

void MathLib::PETScMatrix::add	(	RowColumnIndices< PetscInt > const &	indices,
		const T_DENSE_MATRIX &	sub_matrix )

inline

Add sub-matrix at positions given by global indices, in which negative index indicates ghost entry.

In order to use MatZeroRows to apply Dirichlet boundary condition, entries in the rows with the negative global indices are skipped to added to the global matrix, meanwhile entries in the columns with the negative global indices are added the global matrix. By using MatZeroRows to apply Dirichlet boundary condition, the off diagonal entries in ghost rows of the global matrix are set to zero, while the off diagonal entries in ghost rows of the global matrix are assembled and kept for linear solver.

For the setting of Dirichlet boundary condition in PETSc, please refer to the PETSc:Documentation:FAQ.

Definition at line 146 of file PETScMatrix.h.

    {
        // Set global column indices to positive to allow all entries of columns
        // to be added to the global matrix. For the ghost columns, only the
        // off diagonal entries are added due to the negative indices of the
        // corresponding rows.
        std::vector<PetscInt> cols;
        cols.reserve(indices.columns.size());
        for (auto col : indices.columns)
        {
            // Ghost entries, and its original index is 0.
            if (col == -ncols_)
            {
                cols.push_back(0);
            }
            else
            {
                cols.push_back(std::abs(col));
            }
        }
 
        add(indices.rows, cols, sub_matrix);
    }

References add(), MathLib::RowColumnIndices< IDX_TYPE >::columns, ncols_, and MathLib::RowColumnIndices< IDX_TYPE >::rows.

◆ add() [3/3]

template<class T_DENSE_MATRIX >

void MathLib::PETScMatrix::add	(	std::vector< PetscInt > const &	row_pos,
		std::vector< PetscInt > const &	col_pos,
		const T_DENSE_MATRIX &	sub_mat )

Add a dense sub-matrix to a PETSc matrix.

Parameters

row_pos	The global indices of the rows of the dense sub-matrix.
col_pos	The global indices of the columns of the dense sub-matrix.
sub_mat	A dense sub-matrix to be added. Its data of which must be row oriented stored.

Definition at line 262 of file PETScMatrix.h.

{
    const PetscInt nrows = static_cast<PetscInt>(row_pos.size());
    const PetscInt ncols = static_cast<PetscInt>(col_pos.size());
 
    MatSetValues(A_, nrows, &row_pos[0], ncols, &col_pos[0], &sub_mat(0, 0),
                 ADD_VALUES);
};

References A_.

◆ create()

void MathLib::PETScMatrix::create	(	const PetscInt	d_nz,
		const PetscInt	o_nz )

private

Create the matrix, configure memory allocation and set the related member data.

Parameters

d_nz	Number of nonzeros per row in the diagonal portion of local submatrix (same value is used for all local rows),
o_nz	Number of nonzeros per row in the off-diagonal portion of local submatrix (same value is used for all local rows)

Definition at line 140 of file PETScMatrix.cpp.

{
    MatCreate(PETSC_COMM_WORLD, &A_);
    MatSetSizes(A_, n_loc_rows_, n_loc_cols_, nrows_, ncols_);
 
    MatSetType(A_, MATAIJ);
    MatSetFromOptions(A_);
 
    MatSeqAIJSetPreallocation(A_, d_nz, PETSC_NULLPTR);
    MatMPIAIJSetPreallocation(A_, d_nz, PETSC_NULLPTR, o_nz, PETSC_NULLPTR);
    // If pre-allocation does not work one can use MatSetUp(A_), which is much
    // slower.
 
    MatGetOwnershipRange(A_, &start_rank_, &end_rank_);
    MatGetSize(A_, &nrows_, &ncols_);
    MatGetLocalSize(A_, &n_loc_rows_, &n_loc_cols_);
}

References A_, end_rank_, n_loc_cols_, n_loc_rows_, ncols_, nrows_, and start_rank_.

Referenced by PETScMatrix(), and PETScMatrix().

◆ destroy()

void MathLib::PETScMatrix::destroy ( )

inlineprivate

Definition at line 217 of file PETScMatrix.h.

    {
        if (A_ != nullptr)
        {
            MatDestroy(&A_);
        }
        A_ = nullptr;
    }

References A_.

Referenced by ~PETScMatrix(), and operator=().

◆ finalizeAssembly()

void MathLib::PETScMatrix::finalizeAssembly ( const MatAssemblyType asm_type = MAT_FINAL_ASSEMBLY )

inline

Perform MPI collection of assembled entries in buffer.

Parameters

asm_type Assembly type, either MAT_FLUSH_ASSEMBLY or MAT_FINAL_ASSEMBLY

Definition at line 67 of file PETScMatrix.h.

    {
        MatAssemblyBegin(A_, asm_type);
        MatAssemblyEnd(A_, asm_type);
    }

References A_.

Referenced by MathLib::applyKnownSolution(), MathLib::LinAlg::finalizeAssembly(), and viewer().

◆ getNumberOfColumns()

PetscInt MathLib::PETScMatrix::getNumberOfColumns ( ) const

inline

Get the number of columns.

Definition at line 76 of file PETScMatrix.h.

76{ return ncols_; }

References ncols_.

Referenced by MathLib::addToMatrix(), and MathLib::setMatrix().

◆ getNumberOfLocalColumns()

PetscInt MathLib::PETScMatrix::getNumberOfLocalColumns ( ) const

inline

Get the number of local columns.

Definition at line 80 of file PETScMatrix.h.

80{ return n_loc_cols_; }

References n_loc_cols_.

◆ getNumberOfLocalRows()

PetscInt MathLib::PETScMatrix::getNumberOfLocalRows ( ) const

inline

Get the number of local rows.

Definition at line 78 of file PETScMatrix.h.

78{ return n_loc_rows_; }

References n_loc_rows_.

◆ getNumberOfRows()

PetscInt MathLib::PETScMatrix::getNumberOfRows ( ) const

inline

Get the number of rows.

Definition at line 74 of file PETScMatrix.h.

74{ return nrows_; }

References nrows_.

Referenced by MathLib::addToMatrix(), and MathLib::setMatrix().

◆ getRangeBegin()

PetscInt MathLib::PETScMatrix::getRangeBegin ( ) const

inline

Get the start global index of the rows of the same rank.

Definition at line 82 of file PETScMatrix.h.

82{ return start_rank_; }

References start_rank_.

◆ getRangeEnd()

PetscInt MathLib::PETScMatrix::getRangeEnd ( ) const

inline

Get the end global index of the rows in the same rank.

Definition at line 84 of file PETScMatrix.h.

84{ return end_rank_; }

References end_rank_.

◆ getRawMatrix() [1/2]

Mat & MathLib::PETScMatrix::getRawMatrix ( )

inline

Get matrix reference.

Definition at line 86 of file PETScMatrix.h.

86{ return A_; }

References A_.

Referenced by MathLib::LinAlg::axpy(), MathLib::LinAlg::aypx(), MathLib::LinAlg::matMult(), MathLib::LinAlg::matMultAdd(), MathLib::LinAlg::scale(), and MathLib::PETScLinearSolver::solve().

◆ getRawMatrix() [2/2]

Mat const & MathLib::PETScMatrix::getRawMatrix ( ) const

inline

Get a matrix reference.

Warning: This method is dangerous insofar as you can do arbitrary things also with a const PETSc matrix.

Definition at line 93 of file PETScMatrix.h.

93{ return A_; }

References A_.

◆ operator=()

PETScMatrix & MathLib::PETScMatrix::operator= ( PETScMatrix const & A )

Definition at line 69 of file PETScMatrix.cpp.

{
    nrows_ = A.nrows_;
    ncols_ = A.ncols_;
    n_loc_rows_ = A.n_loc_rows_;
    n_loc_cols_ = A.n_loc_cols_;
    start_rank_ = A.start_rank_;
    end_rank_ = A.end_rank_;
 
    if (A_ != nullptr)
    {
        // TODO this is the slowest option for copying
        MatCopy(A.A_, A_, DIFFERENT_NONZERO_PATTERN);
    }
    else
    {
        destroy();
        MatConvert(A.A_, MATSAME, MAT_INITIAL_MATRIX, &A_);
    }
 
    return *this;
}

References A_, destroy(), end_rank_, n_loc_cols_, n_loc_rows_, ncols_, nrows_, and start_rank_.

◆ set()

void MathLib::PETScMatrix::set	(	const PetscInt	i,
		const PetscInt	j,
		const PetscScalar	value )

inline

Set a single entry with a value.

Parameters

i	The row index.
j	The column index.
value	The entry value.

Definition at line 113 of file PETScMatrix.h.

    {
        MatSetValue(A_, i, j, value, INSERT_VALUES);
    }

References A_.

◆ setRowsColumnsZero()

void MathLib::PETScMatrix::setRowsColumnsZero ( std::vector< PetscInt > const & row_pos )

Set the specified rows to zero except diagonal entries, i.e. \(A(k, j) = \begin{cases} 0.0, &j\not=k, j=1,2,\dots,k-1, k+1, \dots, n \\ 1.0, &j = k \end{cases}\), where \(k \in \mbox{row\_pos}\) This function must be called by all ranks.

Parameters

row_pos The row indices of the specified rows.

Definition at line 92 of file PETScMatrix.cpp.

{
    // Each rank (compute core) processes only the rows that belong to the rank
    // itself.
    const PetscScalar one = 1.0;
    const PetscInt nrows = static_cast<PetscInt>(row_pos.size());
 
    // Each process will only zero its own rows.
    // This avoids all reductions in the zero row routines
    // and thus improves performance for very large process counts.
    // See PETSc doc about MAT_NO_OFF_PROC_ZERO_ROWS.
    MatSetOption(A_, MAT_NO_OFF_PROC_ZERO_ROWS, PETSC_TRUE);
 
    // Keep the non-zero pattern for the assignment operator.
    MatSetOption(A_, MAT_KEEP_NONZERO_PATTERN, PETSC_TRUE);
 
    if (nrows > 0)
    {
        MatZeroRows(A_, nrows, &row_pos[0], one, PETSC_NULLPTR, PETSC_NULLPTR);
    }
    else
    {
        MatZeroRows(A_, 0, PETSC_NULLPTR, one, PETSC_NULLPTR, PETSC_NULLPTR);
    }
}

References A_.

Referenced by MathLib::applyKnownSolution().

◆ setZero()

void MathLib::PETScMatrix::setZero ( )

inline

Set all entries to zero.

Definition at line 95 of file PETScMatrix.h.

95{ MatZeroEntries(A_); }

References A_.

Referenced by MathLib::setMatrix(), and MathLib::setMatrix().

◆ viewer()

void MathLib::PETScMatrix::viewer	(	const std::string &	file_name,
		const PetscViewerFormat	vw_format = PETSC_VIEWER_ASCII_MATLAB )

View the global vector for test purpose. Do not use it for output a big vector.

Parameters

file_name File name for output

vw_format File format listed as: PETSC_VIEWER_DEFAULT Default format PETSC_VIEWER_ASCII_MATLAB MATLAB format PETSC_VIEWER_ASCII_DENSE Print matrix as dense PETSC_VIEWER_ASCII_IMPL Implementation-specific format (which is in many cases the same as the default) PETSC_VIEWER_ASCII_INFO Basic information about object PETSC_VIEWER_ASCII_INFO_DETAIL More detailed info about object PETSC_VIEWER_ASCII_COMMON Identical output format for all objects of a particular type PETSC_VIEWER_ASCII_INDEX (for vectors) Prints the vector element number next to each vector entry PETSC_VIEWER_ASCII_SYMMODU Print parallel vectors without indicating the processor ranges PETSC_VIEWER_ASCII_VTK Outputs the object to a VTK file PETSC_VIEWER_NATIVE Store the object to the binary file in its native format (for example, dense matrices are stored as dense), DMDA vectors are dumped directly to the file instead of being first put in the natural ordering PETSC_VIEWER_DRAW_BASIC Views the vector with a simple 1d plot PETSC_VIEWER_DRAW_LG Views the vector with a line graph PETSC_VIEWER_DRAW_CONTOUR Views the vector with a contour plot

Definition at line 118 of file PETScMatrix.cpp.

{
    PetscViewer viewer;
    PetscViewerASCIIOpen(PETSC_COMM_WORLD, file_name.c_str(), &viewer);
    PetscViewerPushFormat(viewer, vw_format);
 
    finalizeAssembly();
 
    PetscObjectSetName((PetscObject)A_, "Stiffness_matrix");
    MatView(A_, viewer);
 
// This preprocessor is only for debugging, e.g. dump the matrix and exit the
// program.
// #define EXIT_TEST
#ifdef EXIT_TEST
    MatDestroy(A_);
    PetscFinalize();
    exit(0);
#endif
}

References A_, finalizeAssembly(), and viewer().

Referenced by viewer().

Friends And Related Symbol Documentation

◆ finalizeMatrixAssembly

bool finalizeMatrixAssembly	(	PETScMatrix &	mat,
		const MatAssemblyType	asm_type = MAT_FINAL_ASSEMBLY )

friend

General interface for the matrix assembly.

Parameters

mat	The matrix to be finalized.
asm_type	Assembly type, either MAT_FLUSH_ASSEMBLY or MAT_FINAL_ASSEMBLY.

Definition at line 158 of file PETScMatrix.cpp.

{
    mat.finalizeAssembly(asm_type);
    return true;
}

Member Data Documentation

◆ A_

Mat MathLib::PETScMatrix::A_ = nullptr

private

PETSc matrix.

Definition at line 227 of file PETScMatrix.h.

Referenced by PETScMatrix(), add(), add(), create(), destroy(), finalizeAssembly(), getRawMatrix(), getRawMatrix(), operator=(), set(), setRowsColumnsZero(), setZero(), and viewer().

◆ end_rank_

PetscInt MathLib::PETScMatrix::end_rank_

private

Ending index in a rank.

Definition at line 245 of file PETScMatrix.h.

Referenced by create(), getRangeEnd(), and operator=().

◆ n_loc_cols_

PetscInt MathLib::PETScMatrix::n_loc_cols_

private

Number of the local columns.

Definition at line 239 of file PETScMatrix.h.

Referenced by create(), getNumberOfLocalColumns(), and operator=().

◆ n_loc_rows_

PetscInt MathLib::PETScMatrix::n_loc_rows_

private

Number of the local rows.

Definition at line 236 of file PETScMatrix.h.

Referenced by create(), getNumberOfLocalRows(), and operator=().

◆ ncols_

PetscInt MathLib::PETScMatrix::ncols_

private

Number of the global columns.

Definition at line 233 of file PETScMatrix.h.

Referenced by add(), create(), getNumberOfColumns(), and operator=().

◆ nrows_

PetscInt MathLib::PETScMatrix::nrows_

private

Number of the global rows.

Definition at line 230 of file PETScMatrix.h.

Referenced by create(), getNumberOfRows(), and operator=().

◆ start_rank_

PetscInt MathLib::PETScMatrix::start_rank_

private

Starting index in a rank.

Definition at line 242 of file PETScMatrix.h.

Referenced by create(), getRangeBegin(), and operator=().

The documentation for this class was generated from the following files:

MathLib/LinAlg/PETSc/PETScMatrix.h
MathLib/LinAlg/PETSc/PETScMatrix.cpp

Detailed Description

Public Types

Public Member Functions

Private Member Functions

Private Attributes

Friends

Member Typedef Documentation

◆ IndexType

Constructor & Destructor Documentation

◆ PETScMatrix() [1/4]

◆ PETScMatrix() [2/4]

◆ PETScMatrix() [3/4]

◆ ~PETScMatrix()

◆ PETScMatrix() [4/4]

Member Function Documentation

◆ add() [1/3]

◆ add() [2/3]

◆ add() [3/3]

◆ create()

◆ destroy()

◆ finalizeAssembly()

◆ getNumberOfColumns()

◆ getNumberOfLocalColumns()

◆ getNumberOfLocalRows()

◆ getNumberOfRows()

◆ getRangeBegin()

◆ getRangeEnd()

◆ getRawMatrix() [1/2]

◆ getRawMatrix() [2/2]

◆ operator=()

◆ set()

◆ setRowsColumnsZero()

◆ setZero()

◆ viewer()

Friends And Related Symbol Documentation

◆ finalizeMatrixAssembly

Member Data Documentation

◆ A_

◆ end_rank_

◆ n_loc_cols_

◆ n_loc_rows_

◆ ncols_

◆ nrows_

◆ start_rank_