Parallel Interface | Serial Interface | Common Interface |
---|---|---|
sion_paropen_mpi() | sion_open() | sion_ensure_free_space() |
sion_parclose_mpi() | sion_open_rank() | sion_feof() |
sion_close() | sion_bytes_avail_in_block() | |
sion_get_locations() | sion_seek() | |
sion_seek_fp() | ||
sion_fwrite() | ||
sion_fread() |
int sion_paropen_mpi( char *fname, char* file_mode, int *numFiles, MPI_Comm gComm, MPI_Comm *lComm, sion_int64 *chunksize, sion_int32 *fsblksize, int *globalrank, FILE **fileptr, char **newfname); |
||||||||||||||||||||||||||||||||||||||||||||||||||
FSION_PAROPEN_MPI( FNAME, FILE_MODE, NUMFILES, GCOMM, LCOMM, CHUNKSIZE, FSBLKSIZE, GLOBALRANK, NEWFNAME, SID) CHARACTER*(*) FNAME, FILE_MODE, NEWFNAME INTEGER NUMFILES, FSBLKSIZE, GLOBALRANK, SID INTEGER*8 CHUNKSIZE |
||||||||||||||||||||||||||||||||||||||||||||||||||
Description: The function opens on each task one of the physical files with the POSIX open command in direct access mode and sets the file pointer to a task-specific position. The POSIX file pointer will be returned by the parameter fileptr, which can be used to write or read data from file with the POSIX file operations. Important: all parameters of the sion_open function are always pointer. This is because most of the paramater are input values if the file is opened for write and are output parameters if the file is opened fo read. |
Parameters description:
fname : name of file, should equal on all tasks (input) file_mode : like the type parameter of fopen: (input) current recognize option: "rb", "wb" numFiles : number of physical files to use/used (in/out) <=0 groups of tasks write to the same file are described by communicator lComm >0 gComm will be split in numfiles local communicators gComm : MPI communicator, which contains all tasks involved in writing/reading sion file typical: MPI_COMM_WORLD (input) lComm : MPI communicator, which contains all tasks involved in writing/reading to the same physical file typical: MPI_COMM_WORLD (in/out) chunksize : requested space for this task (in/out) fsblcksize : blocksize of filesystem, must be equal on (in/out) all processors, -1 -> use fs default value globalrank : any global unique id for this task, will (in/out) be stored in sion file, usefull if comm is not MPI_COMM_WORLD typical: globalrank= rank in MPI_COMM_WORLD fileptr : filepointer for this task (out) newfname : pointer name of physical file, used by (out) current task, will not returned if newfname is a NULL pointer, will be freed by sion_par_close() |
Return Code:
sid : sion file handle integer (0, ...) -1 if error occured |
int sion_parclose_mpi( int sid ); |
Description: There is currently no fault tolerant handling of the meta-data. The varible size meta-data will be held in memory on each task and will written to disk not before calling this function. This means, that the file cannot be read again if there were problem to write the file to end (like touching the disk space limit). One reason for this, is that all operations between parallel open and close are not collective, which allows to allocate additional chunks for task independently. |
Parameters description:
sid : sion file handle (input) |
Return Code:
rc : 1 if close was ok |
int sion_ensure_free_space(int sid, long numbytes); |
Description: |
Parameters description:
sid : sion file handle (input) numbytes : number of bytes which will be written (input) |
Return Code:
rc : 1 if space could ensured, there is currently no indicator if a new chunk was allocated |
int sion_feof(int sid); |
Description: |
Parameters description:
sid : sion file handle (input) |
Return Code:
rc : 1 if end of last chunk reached 0 otherwise |
long sion_bytes_avail_in_block( int sid ); |
Description: |
Parameters description:
sid : sion file handle (input) |
Return Code:
rc : rc>0 number of bytes rc<=0 file position is after end of block |
int sion_open_rank( char *fname, char *file_mode, long *chunksize, int *fsblocksize, int *rank, FILE **fileptr); |
Description: |
Parameters description:
fname : name of file, should equal on all tasks (input) file_mode : like the type parameter of fopen: (input) current recognize option: "rb", "wb" chunksize : chunksize on this task (in/out) fsblocksize : blocksize of filesystem, must be equal on (in/out) all processors rank : rank number for which the file should be open (in) fileptr : filepointer for this task (out) |
Return Code:
sid : sion file handle integer (0, ...) -1 if error occured |
int sion_open( char *fname, char* file_mode, int *ntasks, int *nfiles, sion_int64 **chunksizes, sion_int32 *fsblksize, int **globalranks, FILE **fileptr); |
Description: In the case of multi-file sion-files sion_open() open internally each physical file with sion_open recursively. sion_open manage for each physical file an own sion file descriptor and data structure. Warning: multi-file-support is not implemented in sion_open of the current version (1.1p9). |
Parameters description:
fname : name of file, should equal on all tasks (input) file_mode : like the type parameter of fopen: (input) current recognize option: "rb", "wb" ntasks : number of tasks used to write this file (in/out) nfiles : number of physical files to be used (in/out) chunksizes : array of chunksizes for each task (in/out) fsblcksize : blocksize of filesystem-1 -> use fs default value (in/out) globalranks : array of global rank number for each task (in/out) fileptr : filepointer for this task (out) |
Return Code:
sid : sion file handle integer (0, ...) -1 if error occured |
int sion_close(int sid); |
Description: |
Parameters description:
sid : sion file handle (input) |
Return Code:
rc : 1 if ok |
int sion_get_locations( int sid, int *size, int *chunks, sion_int64 *globalskip, sion_int64 *start_of_varheader, sion_int64 **sion_localsizes, sion_int64 **sion_globalranks, sion_int64 **sion_chunkcount, sion_int64 **sion_chunksizes); |
Description: |
Parameters description:
sid : sion file handle (input) size : number of tasks wrote to the sion file (out) chunks : maximum number of chunks per tasks used (out) globalskip : distance in bytes between the first bytes of two subsequently chunks of a task (out) start_of_varheader: start position of the meta data block at the end of sion file (out) sion_localsizes: field containing requested chunk size of each task (out) sion_globalranks: field containing global unique id of each task (out) sion_chunkcount: field containing number of chunks used by each task (out) sion_chunksizes: field containing for each tassk and chunk (out) the number of bytes used in this chunk access: sion_chunksizes[rank*chunks+chunknr] |
Return Code:
rc : 1 |
int sion_seek( int sid, int rank, int chunknum, long posinchunk ); |
Description: This function assumes, that the sion file has only one physical files. If the file was created with numfiles>1 the function sion_seek_fp() should be used, which has an additional output parameter fileptr. |
Parameters description:
sid : sion file handle (input) rank : rank number (0,...) (input) SION_CURRENT_RANK to select the current rank chunknum : chunk number (0,...) (input) SION_CURRENT_BLK to select the current block posinchunk : position (0,...) (input) SION_CURRENT_POS to select the current position |
Return Code:
Return Code: 1 if file pointer can be moved to new position 0 otherwise |
int sion_seek( int sid, int rank, int chunknum, long posinchunk, FILE **fileptr ); |
Description: If the sion-file has multiple physical files the function will also change the current files pointer to the file which contains the data of the requested rank. Therefore utilities which uses the serial interface of SIONlib to access sion file should use this function to switch between different tasks of a sion file and should pass the pointer to the POSIX file pointer also to this function. |
Parameters description:
sid : sion file handle (input) rank : rank number (0,...) (input) SION_CURRENT_RANK to select the current rank chunknum : chunk number (0,...) (input) SION_CURRENT_BLK to select the current block posinchunk : position (0,...) (input) SION_CURRENT_POS to select the current position |
Return Code:
Return Code: 1 if file pointer can be moved to new position 0 otherwise |
int sion_fwrite( const void *data, size_t size, size_t n_items, int sid ); |
Description: |
Parameters description:
data : see fwrite documentation size : see fwrite documentation nitems : see fwrite documentation sid : sion file handle (input) |
Return Code:
Return Code: number of elements written (see fwrite documentation) |
int sion_fread( const void *data, size_t size, size_t n_items, int sid ); |
Description: |
Parameters description:
data : see fread documentation size : see fread documentation nitems : see fread documentation sid : sion file handle (input) |
Return Code:
Return Code: number of elements read (see fread documentation) |