Data types and functions for distributed matrices. More...

Data Structures
struct	starneig_distr_block
	Distributed block. More...

Data distributions
enum	starneig_distr_order_t { STARNEIG_ORDER_DEFAULT, STARNEIG_ORDER_ROW_MAJOR, STARNEIG_ORDER_COL_MAJOR }
	Process mapping order. More...

typedef struct starneig_distr *	starneig_distr_t
	Data distribution.

starneig_distr_t	starneig_distr_init ()
	Creates a default data distribution. More...

starneig_distr_t	starneig_distr_init_mesh (int rows, int cols, starneig_distr_order_t order)
	Creates a two-dimensional block cyclic data distribution. More...

starneig_distr_t	starneig_distr_init_func (int(func)(int row, int col, void arg), void *arg, size_t arg_size)
	Creates a distribution using a data distribution function. More...

starneig_distr_t	starneig_distr_duplicate (starneig_distr_t distr)
	Duplicates a data distribution. More...

void	starneig_distr_destroy (starneig_distr_t distr)
	Destroys a data distribution. More...

Distributed matrices
enum	starneig_datatype_t { STARNEIG_REAL_DOUBLE }
	Distributed matrix element data type. More...

typedef struct starneig_distr_matrix *	starneig_distr_matrix_t
	Distributed matrix.

starneig_distr_matrix_t	starneig_distr_matrix_create (int rows, int cols, int row_blksz, int col_blksz, starneig_datatype_t type, starneig_distr_t distr)
	Creates a distributed matrix with uninitialized matrix elements. More...

starneig_distr_matrix_t	starneig_distr_matrix_create_local (int rows, int cols, starneig_datatype_t type, int owner, double *A, int ldA)
	Creates a single-owner distributed matrix from a local matrix. More...

void	starneig_distr_matrix_destroy (starneig_distr_matrix_t matrix)
	Destroys a distributed matrix. More...

void	starneig_distr_matrix_copy (starneig_distr_matrix_t source, starneig_distr_matrix_t dest)
	Copies the contents of a distributed matrix to a second distributed matrix. More...

void	starneig_distr_matrix_copy_region (int sr, int sc, int dr, int dc, int rows, int cols, starneig_distr_matrix_t source, starneig_distr_matrix_t dest)
	Copies region of a distributed matrix to a second distributed matrix. More...

Query functions
void	starneig_distr_matrix_get_blocks (starneig_distr_matrix_t matrix, struct starneig_distr_block *blocks, int num_blocks)
	Returns the locally owned distributed blocks. More...

starneig_distr_t	starneig_distr_matrix_get_distr (starneig_distr_matrix_t matrix)
	Returns the distribution that is associated with a distributed matrix. More...

starneig_datatype_t	starneig_distr_matrix_get_datatype (starneig_distr_matrix_t matrix)
	Returns the matrix element data type. More...

size_t	starneig_distr_matrix_get_elemsize (starneig_distr_matrix_t matrix)
	Returns the matrix element size. More...

int	starneig_distr_matrix_get_rows (starneig_distr_matrix_t matrix)
	Returns the number of (global) rows. More...

int	starneig_distr_matrix_get_cols (starneig_distr_matrix_t matrix)
	Returns the number of (global) columns. More...

int	starneig_distr_matrix_get_row_blksz (starneig_distr_matrix_t matrix)
	Returns the number of rows in a distribution block. More...

int	starneig_distr_matrix_get_col_blksz (starneig_distr_matrix_t matrix)
	Returns the number of columns in a distribution block. More...

Helpers
void	starneig_broadcast (int root, size_t size, void *buffer)
	Broadcast a buffer. More...

Detailed Description

Data types and functions for distributed matrices.

Data Structure Documentation

◆ starneig_distr_block

struct starneig_distr_block

Distributed block.

Data Fields
int	row_blksz	The number of rows in the block.
int	col_blksz	The number of columns in the block.
int	glo_row	The topmost global row that belong to the block.
int	glo_col	The leftmost global column that belong to the block.
int	ld	The leading dimension of the local array.
void *	ptr	A pointer to the local array.

Enumeration Type Documentation

◆ starneig_distr_order_t

enum starneig_distr_order_t

Process mapping order.

Enumerator
STARNEIG_ORDER_DEFAULT	Default ordering.
STARNEIG_ORDER_ROW_MAJOR	Row-major natural ordering.
STARNEIG_ORDER_COL_MAJOR	Column-major natural ordering.

◆ starneig_datatype_t

enum starneig_datatype_t

Distributed matrix element data type.

Enumerator
STARNEIG_REAL_DOUBLE	Double precision real numbers.

Function Documentation

◆ starneig_distr_init()

starneig_distr_t starneig_distr_init ( )

Creates a default data distribution.

Returns: A new data distribution.

◆ starneig_distr_init_mesh()

starneig_distr_t starneig_distr_init_mesh	(	int	rows,
		int	cols,
		starneig_distr_order_t	order
	)

Creates a two-dimensional block cyclic data distribution.

Parameters

[in]	rows	The number of rows in the mesh. Can be set to -1 in which case the library decides the value.
[in]	cols	The number of columns in the mesh. Can be set to -1 in which case the library decides the value.
[in]	order	The process mapping order.

Returns: A new data distribution.

Examples:: gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_init_func()

starneig_distr_t starneig_distr_init_func	(	int()(int row, int col, void arg)	func,
		void *	arg,
		size_t	arg_size
	)

Creates a distribution using a data distribution function.

The distribution function maps each block to it's owner. The function takes three arguments: block's row index, blocks's column index and an optional user defined argument.

struct block_cyclic_arg {
    int rows;
    int cols;
};
int block_cyclic_func(int i, int j, void *arg)
{
    struct block_cyclic_arg *mesh = (struct block_cyclic_arg *) arg;
    return (i % mesh->rows) * mesh->cols + j % mesh->cols;
}
void func(...)
{
    ...
    // create a custom two-dimensional block cyclic distribution with 4 rows
    // and 6 columns in the mesh
    struct block_cyclic_arg arg = { .rows = 4, .cols = 6 };
    starneig_distr_t distr =
        starneig_distr_init_func(&block_cyclic_func, &arg, sizeof(arg));
    ...
}

Parameters

[in]	func	The data distribution function.
[in]	arg	An optional data distribution function argument.
[in]	arg_size	The size of the optional data distribution function argument.

Returns: A new data distribution.

◆ starneig_distr_duplicate()

starneig_distr_t starneig_distr_duplicate ( starneig_distr_t distr )

Duplicates a data distribution.

Parameters

[in] distr The data distribution to be duplicated.

Returns: A duplicated data distribution.

◆ starneig_distr_destroy()

void starneig_distr_destroy ( starneig_distr_t distr )

Destroys a data distribution.

Parameters

[in,out] distr The data distribution to be destroyed.

Examples:: gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_create()

starneig_distr_matrix_t starneig_distr_matrix_create	(	int	rows,
		int	cols,
		int	row_blksz,
		int	col_blksz,
		starneig_datatype_t	type,
		starneig_distr_t	distr
	)

Creates a distributed matrix with uninitialized matrix elements.

// create a m X n double-precision real matrix that is distributed in a
// two-dimensional block cyclic fashion in bm X bn blocks
starneig_distr_t distr = starneig_distr_init();
starneig_distr_matrix_t dA =
    starneig_distr_matrix_create(m, n, bm, bn, STARNEIG_REAL_DOUBLE, distr);

Attention: StarNEig library is designed to use much larger distributed blocks than ScaLAPACK. Selecting a too small distributed block size will be detrimental to the performance.

Parameters

[in]	rows	The number of (global) rows in the matrix.
[in]	cols	The number of (global) columns in the matrix.
[in]	row_blksz	The number of rows in a distribution block. Can be set to -1 in which case the library decides the value.
[in]	col_blksz	The number of columns in a distribution block. Can be set to -1 in which case the library decides the value.
[in]	type	The matrix element data type.
[in]	distr	The data distribution. Can be left to NULL in which case the library decides the distribution.

Returns: A new distributed matrix.

Examples:: gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_create_local()

starneig_distr_matrix_t starneig_distr_matrix_create_local	(	int	rows,
		int	cols,
		starneig_datatype_t	type,
		int	owner,
		double *	A,
		int	ldA
	)

Creates a single-owner distributed matrix from a local matrix.

This creates a wrapper. The contents of the local matrix may be modified by the functions that use the wrapper. The starneig_distr_matrix_destroy() function does not free the local matrix.

int m = 1000, n = 1000;
double *A = NULL; size_t ldA = 0;
// rank 3 initialized the local matrix
if (my_rank = 3) {
    A = initialize_matrix(m, n, &ldA);
}
// all ranks initialize the distributed matrix
starneig_distr_matrix_t lA = starneig_distr_matrix_create_local(
    m, n, STARNEIG_REAL_DOUBLE, 3, A, ldA);

Parameters

[in]	rows	The number of rows in the matrix.
[in]	cols	The number of columns in the matrix.
[in]	type	Matrix element data type.
[in]	owner	MPI rank that owns the distributed matrix.
[in]	A	A pointer to the local matrix. This argument is ignored the calling rank is not the same as the owner.
[in]	ldA	The leading dimension of the local matrix. This argument is ignored the calling rank is not the same as the owner.

Returns: A new distributed matrix.

Examples:: gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_destroy()

void starneig_distr_matrix_destroy ( starneig_distr_matrix_t matrix )

Destroys a distributed matrix.

Parameters

[in,out] matrix The distributed matrix to be destroyed.

Examples:: gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_copy()

void starneig_distr_matrix_copy	(	starneig_distr_matrix_t	source,
		starneig_distr_matrix_t	dest
	)

Copies the contents of a distributed matrix to a second distributed matrix.

Parameters

[in]	source	The source matrix.
[out]	dest	The destination matrix.

Examples:: gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_copy_region()

void starneig_distr_matrix_copy_region	(	int	sr,
		int	sc,
		int	dr,
		int	dc,
		int	rows,
		int	cols,
		starneig_distr_matrix_t	source,
		starneig_distr_matrix_t	dest
	)

Copies region of a distributed matrix to a second distributed matrix.

Parameters

[in]	sr	The first source matrix row to be copied.
[in]	sc	The first source matrix column to be copied.
[in]	dr	The first destination matrix row.
[in]	dc	The first destination matrix column.
[in]	rows	The number of rows to copy.
[in]	cols	The number of columns to copy.
[in]	source	The source matrix.
[out]	dest	The destination matrix.

◆ starneig_distr_matrix_get_blocks()

void starneig_distr_matrix_get_blocks	(	starneig_distr_matrix_t	matrix,
		struct starneig_distr_block **	blocks,
		int *	num_blocks
	)

Returns the locally owned distributed blocks.

Attention: A user is allowed to modify the contents of the locally owned blocks but the the returned array itself should not be modified.

Parameters

[in]	matrix	The distributed matrix.
[out]	blocks	An array that contains all locally owned distributed blocks.
[out]	num_blocks	The total number of locally owned distributed blocks.

◆ starneig_distr_matrix_get_distr()

starneig_distr_t starneig_distr_matrix_get_distr ( starneig_distr_matrix_t matrix )

Returns the distribution that is associated with a distributed matrix.

Attention: The distributed matrix maintains the ownership of the returned data distribution. A user must duplicate the data distribution if necessary.

Parameters