StarNEig Library  version v0.1-beta.2
A task-based library for solving nonsymmetric eigenvalue problems
Distributed Memory / Distributed matrices

Data types and functions for distributed matrices. More...

Data Structures

struct  starneig_distr_block
 Distributed block. More...
 

Data distributions

enum  starneig_distr_order_t { STARNEIG_ORDER_DEFAULT, STARNEIG_ORDER_ROW_MAJOR, STARNEIG_ORDER_COL_MAJOR }
 Process mapping order. More...
 
typedef struct starneig_distr * starneig_distr_t
 Data distribution.
 
starneig_distr_t starneig_distr_init ()
 Creates a default data distribution. More...
 
starneig_distr_t starneig_distr_init_mesh (int rows, int cols, starneig_distr_order_t order)
 Creates a two-dimensional block cyclic data distribution. More...
 
starneig_distr_t starneig_distr_init_func (int(*func)(int row, int col, void *arg), void *arg, size_t arg_size)
 Creates a distribution using a data distribution function. More...
 
starneig_distr_t starneig_distr_duplicate (starneig_distr_t distr)
 Duplicates a data distribution. More...
 
void starneig_distr_destroy (starneig_distr_t distr)
 Destroys a data distribution. More...
 

Distributed matrices

enum  starneig_datatype_t { STARNEIG_REAL_DOUBLE }
 Distributed matrix element data type. More...
 
typedef struct starneig_distr_matrix * starneig_distr_matrix_t
 Distributed matrix.
 
starneig_distr_matrix_t starneig_distr_matrix_create (int rows, int cols, int row_blksz, int col_blksz, starneig_datatype_t type, starneig_distr_t distr)
 Creates a distributed matrix with uninitialized matrix elements. More...
 
starneig_distr_matrix_t starneig_distr_matrix_create_local (int rows, int cols, starneig_datatype_t type, int owner, double *A, int ldA)
 Creates a single-owner distributed matrix from a local matrix. More...
 
void starneig_distr_matrix_destroy (starneig_distr_matrix_t matrix)
 Destroys a distributed matrix. More...
 
void starneig_distr_matrix_copy (starneig_distr_matrix_t source, starneig_distr_matrix_t dest)
 Copies the contents of a distributed matrix to a second distributed matrix. More...
 
void starneig_distr_matrix_copy_region (int sr, int sc, int dr, int dc, int rows, int cols, starneig_distr_matrix_t source, starneig_distr_matrix_t dest)
 Copies region of a distributed matrix to a second distributed matrix. More...
 

Query functions

void starneig_distr_matrix_get_blocks (starneig_distr_matrix_t matrix, struct starneig_distr_block **blocks, int *num_blocks)
 Returns the locally owned distributed blocks. More...
 
starneig_distr_t starneig_distr_matrix_get_distr (starneig_distr_matrix_t matrix)
 Returns the distribution that is associated with a distributed matrix. More...
 
starneig_datatype_t starneig_distr_matrix_get_datatype (starneig_distr_matrix_t matrix)
 Returns the matrix element data type. More...
 
size_t starneig_distr_matrix_get_elemsize (starneig_distr_matrix_t matrix)
 Returns the matrix element size. More...
 
int starneig_distr_matrix_get_rows (starneig_distr_matrix_t matrix)
 Returns the number of (global) rows. More...
 
int starneig_distr_matrix_get_cols (starneig_distr_matrix_t matrix)
 Returns the number of (global) columns. More...
 
int starneig_distr_matrix_get_row_blksz (starneig_distr_matrix_t matrix)
 Returns the number of rows in a distribution block. More...
 
int starneig_distr_matrix_get_col_blksz (starneig_distr_matrix_t matrix)
 Returns the number of columns in a distribution block. More...
 

Helpers

void starneig_broadcast (int root, size_t size, void *buffer)
 Broadcast a buffer. More...
 

Detailed Description

Data types and functions for distributed matrices.


Data Structure Documentation

◆ starneig_distr_block

struct starneig_distr_block

Distributed block.

Data Fields
int row_blksz The number of rows in the block.
int col_blksz The number of columns in the block.
int glo_row The topmost global row that belong to the block.
int glo_col The leftmost global column that belong to the block.
int ld The leading dimension of the local array.
void * ptr A pointer to the local array.

Enumeration Type Documentation

◆ starneig_distr_order_t

Process mapping order.

Enumerator
STARNEIG_ORDER_DEFAULT 

Default ordering.

STARNEIG_ORDER_ROW_MAJOR 

Row-major natural ordering.

STARNEIG_ORDER_COL_MAJOR 

Column-major natural ordering.

◆ starneig_datatype_t

Distributed matrix element data type.

Enumerator
STARNEIG_REAL_DOUBLE 

Double precision real numbers.

Function Documentation

◆ starneig_distr_init()

starneig_distr_t starneig_distr_init ( )

Creates a default data distribution.

Returns
A new data distribution.

◆ starneig_distr_init_mesh()

starneig_distr_t starneig_distr_init_mesh ( int  rows,
int  cols,
starneig_distr_order_t  order 
)

Creates a two-dimensional block cyclic data distribution.

Parameters
[in]rowsThe number of rows in the mesh. Can be set to -1 in which case the library decides the value.
[in]colsThe number of columns in the mesh. Can be set to -1 in which case the library decides the value.
[in]orderThe process mapping order.
Returns
A new data distribution.
Examples:
gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_init_func()

starneig_distr_t starneig_distr_init_func ( int(*)(int row, int col, void *arg)  func,
void *  arg,
size_t  arg_size 
)

Creates a distribution using a data distribution function.

The distribution function maps each block to it's owner. The function takes three arguments: block's row index, blocks's column index and an optional user defined argument.

struct block_cyclic_arg {
int rows;
int cols;
};
int block_cyclic_func(int i, int j, void *arg)
{
struct block_cyclic_arg *mesh = (struct block_cyclic_arg *) arg;
return (i % mesh->rows) * mesh->cols + j % mesh->cols;
}
void func(...)
{
...
// create a custom two-dimensional block cyclic distribution with 4 rows
// and 6 columns in the mesh
struct block_cyclic_arg arg = { .rows = 4, .cols = 6 };
starneig_distr_init_func(&block_cyclic_func, &arg, sizeof(arg));
...
}
Parameters
[in]funcThe data distribution function.
[in]argAn optional data distribution function argument.
[in]arg_sizeThe size of the optional data distribution function argument.
Returns
A new data distribution.

◆ starneig_distr_duplicate()

starneig_distr_t starneig_distr_duplicate ( starneig_distr_t  distr)

Duplicates a data distribution.

Parameters
[in]distrThe data distribution to be duplicated.
Returns
A duplicated data distribution.

◆ starneig_distr_destroy()

void starneig_distr_destroy ( starneig_distr_t  distr)

Destroys a data distribution.

Parameters
[in,out]distrThe data distribution to be destroyed.
Examples:
gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_create()

starneig_distr_matrix_t starneig_distr_matrix_create ( int  rows,
int  cols,
int  row_blksz,
int  col_blksz,
starneig_datatype_t  type,
starneig_distr_t  distr 
)

Creates a distributed matrix with uninitialized matrix elements.

// create a m X n double-precision real matrix that is distributed in a
// two-dimensional block cyclic fashion in bm X bn blocks
Attention
StarNEig library is designed to use much larger distributed blocks than ScaLAPACK. Selecting a too small distributed block size will be detrimental to the performance.
Parameters
[in]rowsThe number of (global) rows in the matrix.
[in]colsThe number of (global) columns in the matrix.
[in]row_blkszThe number of rows in a distribution block. Can be set to -1 in which case the library decides the value.
[in]col_blkszThe number of columns in a distribution block. Can be set to -1 in which case the library decides the value.
[in]typeThe matrix element data type.
[in]distrThe data distribution. Can be left to NULL in which case the library decides the distribution.
Returns
A new distributed matrix.
Examples:
gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_create_local()

starneig_distr_matrix_t starneig_distr_matrix_create_local ( int  rows,
int  cols,
starneig_datatype_t  type,
int  owner,
double *  A,
int  ldA 
)

Creates a single-owner distributed matrix from a local matrix.

This creates a wrapper. The contents of the local matrix may be modified by the functions that use the wrapper. The starneig_distr_matrix_destroy() function does not free the local matrix.

int m = 1000, n = 1000;
double *A = NULL; size_t ldA = 0;
// rank 3 initialized the local matrix
if (my_rank = 3) {
A = initialize_matrix(m, n, &ldA);
}
// all ranks initialize the distributed matrix
m, n, STARNEIG_REAL_DOUBLE, 3, A, ldA);
Parameters
[in]rowsThe number of rows in the matrix.
[in]colsThe number of columns in the matrix.
[in]typeMatrix element data type.
[in]ownerMPI rank that owns the distributed matrix.
[in]AA pointer to the local matrix. This argument is ignored the calling rank is not the same as the owner.
[in]ldAThe leading dimension of the local matrix. This argument is ignored the calling rank is not the same as the owner.
Returns
A new distributed matrix.
Examples:
gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_destroy()

void starneig_distr_matrix_destroy ( starneig_distr_matrix_t  matrix)

Destroys a distributed matrix.

Parameters
[in,out]matrixThe distributed matrix to be destroyed.
Examples:
gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_copy()

void starneig_distr_matrix_copy ( starneig_distr_matrix_t  source,
starneig_distr_matrix_t  dest 
)

Copies the contents of a distributed matrix to a second distributed matrix.

Parameters
[in]sourceThe source matrix.
[out]destThe destination matrix.
Examples:
gep_dm_full_chain.c, and sep_dm_full_chain.c.

◆ starneig_distr_matrix_copy_region()

void starneig_distr_matrix_copy_region ( int  sr,
int  sc,
int  dr,
int  dc,
int  rows,
int  cols,
starneig_distr_matrix_t  source,
starneig_distr_matrix_t  dest 
)

Copies region of a distributed matrix to a second distributed matrix.

Parameters
[in]srThe first source matrix row to be copied.
[in]scThe first source matrix column to be copied.
[in]drThe first destination matrix row.
[in]dcThe first destination matrix column.
[in]rowsThe number of rows to copy.
[in]colsThe number of columns to copy.
[in]sourceThe source matrix.
[out]destThe destination matrix.

◆ starneig_distr_matrix_get_blocks()

void starneig_distr_matrix_get_blocks ( starneig_distr_matrix_t  matrix,
struct starneig_distr_block **  blocks,
int *  num_blocks 
)

Returns the locally owned distributed blocks.

Attention
A user is allowed to modify the contents of the locally owned blocks but the the returned array itself should not be modified.
Parameters
[in]matrixThe distributed matrix.
[out]blocksAn array that contains all locally owned distributed blocks.
[out]num_blocksThe total number of locally owned distributed blocks.

◆ starneig_distr_matrix_get_distr()

starneig_distr_t starneig_distr_matrix_get_distr ( starneig_distr_matrix_t  matrix)

Returns the distribution that is associated with a distributed matrix.

Attention
The distributed matrix maintains the ownership of the returned data distribution. A user must duplicate the data distribution if necessary.
Parameters
[in]matrixThe distributed matrix.
Returns
The associated distribution.

◆ starneig_distr_matrix_get_datatype()

starneig_datatype_t starneig_distr_matrix_get_datatype ( starneig_distr_matrix_t  matrix)

Returns the matrix element data type.

Parameters
[in]matrixThe distributed matrix.
Returns
The matrix element data type.

◆ starneig_distr_matrix_get_elemsize()

size_t starneig_distr_matrix_get_elemsize ( starneig_distr_matrix_t  matrix)

Returns the matrix element size.

Parameters
[in]matrixThe distributed matrix.
Returns
The matrix element size.

◆ starneig_distr_matrix_get_rows()

int starneig_distr_matrix_get_rows ( starneig_distr_matrix_t  matrix)

Returns the number of (global) rows.

Parameters
[in]matrixThe distributed matrix.
Returns
The number of (global) rows.

◆ starneig_distr_matrix_get_cols()

int starneig_distr_matrix_get_cols ( starneig_distr_matrix_t  matrix)

Returns the number of (global) columns.

Parameters
[in]matrixThe distributed matrix.
Returns
The number of (global) columns.

◆ starneig_distr_matrix_get_row_blksz()

int starneig_distr_matrix_get_row_blksz ( starneig_distr_matrix_t  matrix)

Returns the number of rows in a distribution block.

Parameters
[in]matrixThe distributed matrix.
Returns
The number of rows in a distribution block.

◆ starneig_distr_matrix_get_col_blksz()

int starneig_distr_matrix_get_col_blksz ( starneig_distr_matrix_t  matrix)

Returns the number of columns in a distribution block.

Parameters
[in]matrixThe distributed matrix.
Returns
The number of columns in a distribution block.

◆ starneig_broadcast()

void starneig_broadcast ( int  root,
size_t  size,
void *  buffer 
)

Broadcast a buffer.

Parameters
[in]rootThe rank that is going to broadcast the buffer.
[in]sizeThe size of the buffer.
[in,out]bufferA pointer to the buffer.