The `prosimu` simulation reader¶

The prosimu class is designed to read netCDF files used in simulations (inputs, outputs). It is still maintained but new work should be done with xarray

Created on 4 oct. 2012

Authors:

lafaysse
LVG

Introduction¶

This module contains the prosimu class used to read simulation files (netCDF format) as produced by SURFEX/Crocus for instance.

More in details, prosimu1d and prosimu2d classes are provided to read 1D files (ony one spatial dimension, Number_of_points) or 2D files (gridded simulations with x and y axes). Unless your application is specific to 1D or 2D files, please use prosimu_auto that will instantiate the appropriate class (prosimu1d or prosimu2d).

Examples¶

A short example of how to use this module and read time dimension and snow density variables (RSN_VEG) for all layers at point in massif 11, 2700m, northern aspect on 40 degree slope:

from snowtools.utils.prosimu import prosimu_auto

with prosimu_auto('/path/to/my/PRO_XXX.nc') as ff:
        time= ff.readtime()
        point = ff.get_point(ZS=2700, aspect=0, slope=40, massif_num=11)
        density = ff.read('RSN_VEG', selectpoint=point)

time and density now contains the data in the form of numpy arrays for the selected point. As RSN_VEG is a variable available for all layers, the density array is a 3D numpy ndarray with dimensions (time, layers).

A short example to get multiple points at once:

from snowtools.utils.prosimu import prosimu_auto

with prosimu_auto('/path/to/my/PRO_XXX.nc') as ff:
        time= ff.readtime()
        points = ff.get_points(ZS=2700, slope=40, massif_num=11)
        density = ff.read('RSN_VEG', selectpoint=points)

time and density now contains the data in the form of numpy arrays for the selected points (all aspects). The density is a 3D numpy array with first dimension being time, second dimension being layers and the last one is points.

Example to get total snow water equivalent of the snowpack (variable WSN_T_ISBA) across time for all points:

from snowtools.utils.prosimu import prosimu_auto

with prosimu_auto('/path/to/my/PRO_XXX.nc') as ff:
        time= ff.readtime()
        swe_total = ff.read('WSN_T_ISBA')

Main methods¶

Most useful methods are:

prosimuAbstract.gettime() to read tim dimension
prosimuAbstract.listvar() to have a list of available variables
prosimuAbstract.read() to read data from one variable
prosimuAbstract.get_point() to help in selecting a point across spatial dimensions

utils.prosimu.prosimu¶: alias of prosimu1d

class utils.prosimu.prosimu1d(path, ncformat='NETCDF3_CLASSIC', openmode='r')[source]¶

Bases: _prosimu1d2d, prosimuAbstract

Class to read simulations where simulation points are aggregated along one dimension. This is commonly used for semi-distributed simulations.

forceread : si True force lors de la lecture d’une variable par la méthode read_var à mettre l’ensemble des données de cette variable en cache - utile lorsque de grands nombre d’accès à la même variable sont nécessaires

class utils.prosimu.prosimu2d(path, ncformat='NETCDF3_CLASSIC', openmode='r')[source]¶

Bases: _prosimu1d2d, prosimuAbstract

Class to read simulations where simulation points are aggregated along one dimension. This is commonly used for semi-distributed simulations.

forceread : si True force lors de la lecture d’une variable par la méthode read_var à mettre l’ensemble des données de cette variable en cache - utile lorsque de grands nombre d’accès à la même variable sont nécessaires

class utils.prosimu.prosimuAbstract(path, ncformat='NETCDF3_CLASSIC', openmode='r')[source]¶

Bases: ABC

Abstract class designed to read simulations files Describe the interface provided to access simulations.

Parameters:

path (path-like or list) – path of the file to read
ncformat (str) – NetCDF format to use.
openmode (str) – open mode (mainly r, w or r+)

Do not forget to close the file at the end or use a context manager:

with prosimu(filename) as ff:
    time= ff.readtime()
    var = ff.read(varname)
    # Do your stuff

To read data (in variables), see read() or read_var()

Note that the recommended file format is NETCDF4_CLASSIC. Prosimu also allow for using NETCDF3_CLASSIC. However, NETCDF4 is discouraged because this format is not supported by netCDF4.MFDataset function that is used to open several files at one time.

path could be a list of files to concatenate along an undefined length dimension (should be unique, e.g. time dimension). Note that in this case, NETCDF4 format is highly discouraged as it implies a full type conversion and copy of the data.

path could also be a string with a glob sign * to automatically fill a list of files to concatenate.

Please note that in case of opening simultaneously several files, only the 'r' open mode is allowed.

forceread : si True force lors de la lecture d’une variable par la méthode read_var à mettre l’ensemble des données de cette variable en cache - utile lorsque de grands nombre d’accès à la même variable sont nécessaires

close()[source]¶: Close the netCDF dataset.

force_read_in_cache()[source]¶

Force la lecture des variables du netcdf sous forme de tableau numpy. Ces tableaux sont stockés dans l’attribut varcache de la classe

Le cache est utilisé par la méthode read_var, son utilisation n’est pas implémentée pour les autres méthodes

Utile lorsque de nombreuses lectures de la même variable sont requises

format()[source]¶

Get the format of the undelying netCDF file (e.g. NETCDF3_CLASSIC)

Returns:: The format of NetCDF file
Return type:: str

get_point(**kwargs)[source]¶: Same as get_points() but raise an exception if there is not exactly one point.

abstract get_points(**kwargs)[source]¶

Return the values of dimension number of points correpsonding to a subset defined by variables such as aspect, elevation (ZS), massif_num, slope, latitude, longitude according to named arguments passed to the function.

Returns:: List of points
Return type:: list

get_time(time_asdatetime)[source]¶: Renvoie l’indice de la dimension time correspondant au datetime donné en argument

getattr(varname, attname)[source]¶

Get the value of an attribute of a variable

Parameters:

varname (str) – the variable name
attname (str) – the attribute name

Returns:

the attribute value

getdimvar(varname)[source]¶

The dimensions of a specific variable

Parameters:: varname (str) – the variable name
Returns:: dimensions of varname
Return type:: numpy array

getfillvalue(varname)[source]¶

Get the fill value for a variable

Parameters:: varname (str) – Variable name
Returns:: the fill value

getlendim(dimname)[source]¶

Return the length of dimension dimname

Returns:: length of dimension dimname
Return type:: int

getrankvar(varname)[source]¶

Get the rank (number of dimensions) of a variable

Parameters:: varname (str) – the variable name
Returns:: rank
Return type:: int

gettiledim()[source]¶

Return the number of tile in the file or None if no tile dimension is found.

Returns:: Number of tile
Return type:: int or None

gettypevar(varname)[source]¶

Return the dtype of a variable

Parameters:: varname (str) – the variable name
Returns:: the corresponding dtype

integration(variable, nstep, start=0)[source]¶: Renvoie les valeurs cumulées tous les nstep pas de temps

Todo

Documenter cette fonction

listattr(varname)[source]¶

Get the list of attributtes attached to a variable

Parameters:: varname (str) – the variable name
Returns:: atributes

listdim()[source]¶: Return a copy of the list of dimensions present in the netCDF file

listvar()[source]¶

Return the list of variables present in the netCDF file

Returns:: list of variables
Return type:: list

moytempo(precip, nstep, start=0)[source]¶: Même chose que integration mais renvoie une moyenne

Todo

Documenter cette fonction

abstract read(varname, fill2zero=False, selectpoint=None, keepfillvalue=False, tile=None, removetile=True, needmodif=False, hasDecile=False)[source]¶

Read data from a variable in the netCDF file.

Parameters:

varname (str) – the variable name
fill2zero (bool) – if True, will replace undefined data with the value of 0. Otherwise, np.nan is used except if keepfillvalue is set.
selectpoint (integer or list of integers) – Select a point. If -1 or None, select all points.
tile (int) – Select a tile.
keepfillvalue (bool) – Do not replace the undefined data with np.nan and keep the fill value used in the netCDF file.
needmodif (bool) – If True, return also the variable object
hasDecile – Deprecated, please do not use.
removetile – Deprecated, see tile instead.

Returns:

Numpy data array containing the data of the selected variable. Note that order of dimensions is the same as in in file except if selectpoint is a list. In this case, the resulting point dimension is appended as the last dimension.

Return type:

numpy array

read_var(variable_name, **kwargs)[source]¶

Read a variable from netCDF file. Note that this function does not support patches.

Parameters:

variable_name – nom de la variable
kwargs – spécifier la sous-sélection sous la forme dimname = value ou dimname est le nom de la dimension d’intéret, value est une valeur numérique ou un objet slice python pour récupérer une plage de valeurs

Returns:

un tableau numpy.ma.MaskedArray (on peut toujours remplacer les éléments masqués par un indicateur de valeur manquante pas implémenté)

Exemples:

snowemp = prosimu.read_var('SNOWTEMP', time=0, Number_of_points = slice(100,125))
snowtemp = prosimu.read_var('SNOWTEMP', time=slice(0,10,2), Number_of_points=1, snow_layer=slice(0,10))

peut-être utilisé en combinaison avec les méthodes get_point et get_time pour récupérer un point / un instant donné :

snowtemp = prosimu.read_var('SNOWTEMP', time=self.get_time(datetime(2018,3,1,9)),
            Number_of_points = self.get_point(massif_num=3,slope=20,ZS=4500,aspect=0))

readtime()[source]¶

Get the time dimension of the netCDF file

Returns:: time axis data
Return type:: numpy array

readtime_for_copy()[source]¶

Get the time raw variable and units

Returns:: time variable, time units

utils.prosimu.prosimu_auto(path, ncformat='NETCDF3_CLASSIC', openmode='r', **kwargs)[source]¶

Factory function that guess the correct class to return among prosimu1d or prosimu2d.

Parameters:

path (str or path-like or list) – file path or file path list
openmode (str) – Open mode (r, r+ or w generally)
kwargs – Other arguments passed to the instantiated class

Ncformat:

netCDF format only in case of creation

class utils.prosimu.prosimu_base(path, ncformat='NETCDF3_CLASSIC', openmode='r')[source]¶

Bases: _prosimu1d2d, prosimuAbstract

Common class that is allowed to read both 1d and 2d simualtions. The selection of points is not allowed in this class.

forceread : si True force lors de la lecture d’une variable par la méthode read_var à mettre l’ensemble des données de cette variable en cache - utile lorsque de grands nombre d’accès à la même variable sont nécessaires

get_points(**kwargs)[source]¶

class utils.prosimu.prosimu_old(path, ncformat='NETCDF3_CLASSIC', openmode='r')[source]¶

Bases: prosimu1d

In the old operationnal format (before 2018), some dimensions have different names, this class allows to deal with them easily

Warning

Should not be used nowadays

forceread : si True force lors de la lecture d’une variable par la méthode read_var à mettre l’ensemble des données de cette variable en cache - utile lorsque de grands nombre d’accès à la même variable sont nécessaires

The `prosimu` simulation reader¶

Introduction¶

Examples¶

Main methods¶

Table of Contents

Previous topic

Next topic

This Page

The prosimu simulation reader¶

Introduction¶

Examples¶

Main methods¶

The `prosimu` simulation reader¶