Plots from operational simulations

Created on 6 déc. 2018

@author: lafaysse

usage: python postprocess.py [-b YYYYMMDD] [-e YYYYMMDD] [-o diroutput]

1. extracts operational simulation results

2. Plots maps for the Alps, the Pyrenees the Corse, Vosges, Massif Central, Jura

3. Creates spaghetti plots for all massifs and stations

class plots.pearps2m.postprocess.Ensemble

Bases: object

Describes an ensemble of simulations

__init__()

close()

Close simulation files and remove data from ensemble.

ensemble

data dict with variable names as keys and np.arrays as values

get_alti()

Get altitude variable from the first simulation file.

Returns:

altitude variable

Return type:

numpy array

get_aspect()

Get aspect variable from the first simulation file.

Returns:

aspect variable

Return type:

numpy array

get_metadata()

Get a tuple with spatial indices.

Returns:

indpoints, indpoints

get_npoints()

Get the number of spatial points.

Returns:

total number of spatial points

Return type:

int

mean(varname)

Calculate ensemble mean for a given variable.

Parameters:

varname (str) – variable name

Returns:

ensemble mean

Return type:

numpy array

open(listmembers)

Opens simulation files

Parameters:

listmembers (list) – list of ensemble members (filenames)

Variables:

• simufiles (list) – list of prosimu objects with simulation file

• inddeterministic (int) – index of the deterministic member

• nmembers – number of members

• ~.time (numpy array) – time variable from the first simulation file

• indpoints (list) – list with spatial indices

• npoints (int) – total number of spatial points

• nech (int) – number of time steps (forecast steps)

probability(varname, seuilinf=-999999999, seuilsup=999999999)

Calculates probability as the proportion of ensemble members with values larger than seuilinf and smaller than seuilsup.

Parameters:

• varname (str) – name of the variable for which to calculate the probability

• seuilinf – lower threshold

• seuilsup – upper threshold

Returns:

probability field (or np.nan if the ensemble is not defined)

Return type:

numpy array

quantile(varname, level)

Calculates ensemble percentiles for a given variable and given percentile levels.

Parameters:

• varname (str) – Variable name

• level (list of numbers between 0 and 100) – list of percentiles to calculate

Returns:

array of percentiles

Return type:

numpy array

read(varname)

Read a variable and store it in ensemble

Parameters:

varname (str) – name of the variable to be read (corresponds to the variable name of the simulation NetCDF files)

read_geovar(varname)

Read a variable from the first simulation file.

Parameters:

varname (str) – NetCDF variable name of the variable to read

Returns:

data read

Return type:

numpy array

select_points()

Get a list of spatial indices from the spatial dimension length in the first simulation file.

Returns:

list of spatial points (indices)

Return type:

list

property spatialdim

Name of spatial dimension

spread(varname)

Calculate ensemble spread (standard deviation) for a given variable.

Parameters:

varname (str) – variable name

Returns:

ensemble spread

Return type:

numpy array

Return type:

numpy array

class plots.pearps2m.postprocess.EnsembleDiags

Bases: Ensemble

Probabilities and quantiles from ensembles.

__init__()

close()

Close simulation files and remove data from ensemble, proba and quantiles

diags(list_var, list_quantiles, list_seuils)

Calculate quantiles and exceedance probabilities for a list of variables, quantiles and thresholds. The quantiles are stored in quantiles and the probabilities in proba

Parameters:

• list_var (iterable) – list of variables

• list_quantiles (iterable) – list of percentiles to be calculated

• list_seuils (dict of iterables with variable names as keys) – list of thresholds for each variable

proba

probabilities

quantiles

percentiles

class plots.pearps2m.postprocess.EnsembleFlatMassif

Bases: _EnsembleMassif

Class for ensemble simulations on a massif geometry (SAFRAN like) where all data points are considered on flat terrain (zero slope and no orientation information).

select_points()

Select spatial indices from the first simulation file where aspect=-1 (zero slope, no orientation information).

Returns:

spatial indices to be read from simulation files

Return type:

numpy boolean array

class plots.pearps2m.postprocess.EnsembleHydro(variables, inputfiles, decile=range(10, 100, 10), outdir='.')

Bases: EnsemblePostproc

Class to provide a synthesis of ensemble hydrological diagnostics of S2M

Parameters:

• variables – list of variables to process

• inputfiles – list of input files

• decile – list of percentiles

• outdir (str) – Output directory

property spatialdim

Name of spatial dimension

property standardvars

variables always written to the output file

class plots.pearps2m.postprocess.EnsembleMassifPoint(massif_num, alti, aspect, slope)

Bases: _EnsembleMassif

Class for extracting one specific point in massif geometry

__init__(massif_num, alti, aspect, slope)

select_points()

Select index of the corresponding point

class plots.pearps2m.postprocess.EnsembleNorthSouthMassif

Bases: _EnsembleMassif

Class for ensemble simulations on a massif geometry (SAFRAN like) where data points are considered at a slope of 40 degrees and two orientations: North and South.

select_points()

Select spatial indices from the first simulation file where slope=40 and aspect either 0 or 180.

Returns:

spatial indices to be read from simulation files

Return type:

a tuple of numpy boolean array, where the first component corresponds to the Northern orientation and the second to the Southern one.

class plots.pearps2m.postprocess.EnsembleOperDiags

Bases: EnsembleDiags

Class for operationally used plots.

__init__()

alldiags()

Calculate percentiles for all variables in list_var_spag and list_var_map.

attributes = {'DSN_T_ISBA': {'convert_unit': 100.0, 'label': 'Hauteur de neige (cm)'}, 'NAT_LEV': {'forcemax': 5.5, 'forcemin': -0.5, 'label': 'Risque naturel', 'ncolors': 6, 'palette': 'YlOrRd', 'ticks': ['Très faible', 'Faible', 'Mod. A', 'Mod. D', 'Fort', 'Très fort']}, 'PP_SD_1DY_ISBA': {'convert_unit': 1.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Epaisseur de neige fraîche en 24h (cm)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'RAINF_ISBA': {'convert_unit': 10800.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Pluie en 3h (kg/m2/3h)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'RAMSOND_ISBA': {'convert_unit': 100.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Epaisseur mobilisable (cm)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'REFRZTH_ISBA': {'convert_unit': 100.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Epaisseur regelée (cm)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'SD_1DY_ISBA': {'convert_unit': 100.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Epaisseur de neige fraîche en 24h (cm)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'SD_3DY_ISBA': {'convert_unit': 100.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Epaisseur de neige fraîche en 72h (cm)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'SNOMLT_ISBA': {'convert_unit': 10800.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Ecoulement en 3h (kg/m2/3h)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'WET_TH_ISBA': {'convert_unit': 100.0, 'forcemax': 60.0, 'forcemin': 0.0, 'label': 'Epaisseur humide (cm)', 'palette': 'YlGnBu', 'seuiltext': 50.0}, 'WSN_T_ISBA': {'label': 'Equivalent en eau (kg/m2)'}, 'naturalIndex': {'forcemax': 8.0, 'forcemin': 0.0, 'format': '%.1f', 'label': 'Indice de risque naturel', 'nolevel': True, 'palette': 'YlOrRd'}}

dict of plot attributes for each variable

formatplot = 'png'

plot format

list_q = [20, 50, 80]

list of quantiles

pack_spaghettis(suptitle, diroutput='.')

Produce spaghetti plots for all variables in list_var_spag.

Parameters:

• suptitle (unicode string) – Suptitle for all plots.

• diroutput – directory to save the plots

pack_spaghettis_multipoints(list_pairs, suptitle, diroutput='.', **kwargs)

Produce spaghetti plots with up to four points per plot for variables in list_var_spag_2points.

Each point gets a different color.

Parameters:

• list_pairs (list of list) – list of indices to plot together on each plot

• suptitle (unicode str) – common suptitle for all plots

• diroutput – output directory

• kwargs – arguments to be passed to plotting routines, notably “labels”

class plots.pearps2m.postprocess.EnsembleOperDiagsFlatMassif

Bases: EnsembleOperDiags, EnsembleFlatMassif

Class for operationally plotting maps and spaghetti plots for ensembles with massif geometry and for the zero slope case.

levelmax = 3900

maximum height level to be treated

levelmin = 0

minimum heigth level to be treated

list_var_map = ['naturalIndex', 'SD_1DY_ISBA', 'SD_3DY_ISBA', 'SNOMLT_ISBA']

list of variables to plot maps for

list_var_spag = ['naturalIndex', 'DSN_T_ISBA', 'WSN_T_ISBA', 'SNOMLT_ISBA']

list of variables to do spaghetti plots for

pack_maps(domain, suptitle, diroutput='.')

Produce maps for the variables given in list_var_map over the regions given in domain for each available altitude level and time step.

Each massif is colored corresponding to the second percentile value defined in list_q and the color map defined in attributes. At the center of each massif the values corresponding to the first three percentiles in list_q are marked.

Parameters:

• domain – list of region identifiers (e.g., “alp”, “pyr”, “cor”)

• suptitle (str) – common suptitle for all plots

• diroutput – output directory to save the maps

class plots.pearps2m.postprocess.EnsembleOperDiagsNorthSouthMassif

Bases: EnsembleOperDiags, EnsembleNorthSouthMassif

Class for operationally plot maps and spaghetti plots distinguishing between northern and southern orientation at each massif.

alldiags()

Calculate percentiles for all variables in list_var_spag_2points and list_var_map.

ensemble = {}

ensemble data

get_pairs_ns()

Get for each massif and altitude the pair of indices corresponding to values for the northern and southern slopes.

Returns:

indices corresponding to northern and southern slopes

Return type:

list of lists

levelmax = 3900

maximum height level to plot

levelmin = 0

minimum height level to plot

list_var_map = ['RAMSOND_ISBA', 'NAT_LEV', 'WET_TH_ISBA', 'REFRZTH_ISBA']

variable list for producing maps

list_var_spag = []

variable list for standard spaghetti plots

list_var_spag_2points = ['RAMSOND_ISBA', 'NAT_LEV', 'WET_TH_ISBA', 'REFRZTH_ISBA']

variable list for spaghetti plots combining northern and southern orientation

pack_maps(domain, suptitle, diroutput)

Produce maps for the variables given in list_var_map over the regions given in domain for each available altitude level and time step.

For each massif a table with background colors and values corresponding to the percentiles in list_q and the color map defined in attributes is plotted near the center of each massif.

Parameters:

• domain – list of region identifiers (e.g., “alp”, “pyr”, “cor”)

• suptitle (str) – common suptitle for all plots

• diroutput – output directory to save the maps

pack_spaghettis_ns(suptitle, diroutput='.')

Do spaghetti plots with values for the northern and southern slope on the same plot.

Parameters:

• suptitle – common suptitle for all plots

• diroutput – directory to save the plots

versants = ['Nord 40°', 'Sud 40°']

labels for the two orientations

class plots.pearps2m.postprocess.EnsembleOperDiagsStations

Bases: EnsembleOperDiags, EnsembleStation

Class for operationally plotting spaghetti plots for station based simulations.

list_var_map = []

list of variables to plot maps for

list_var_spag = ['DSN_T_ISBA', 'WSN_T_ISBA', 'RAMSOND_ISBA', 'WET_TH_ISBA', 'REFRZTH_ISBA', 'SNOMLT_ISBA']

list of variables to plot spaghetti plots for.

class plots.pearps2m.postprocess.EnsemblePostproc(variables, inputfiles, decile=range(10, 100, 10), outdir='.')

Bases: _EnsembleMassif

Class for ensemble post-processing.

Parameters:

• variables – list of variables to process

• inputfiles – list of input files

• decile – list of percentiles

• outdir (str) – Output directory

__init__(variables, inputfiles, decile=range(10, 100, 10), outdir='.')

Parameters:

• variables – list of variables to process

• inputfiles – list of input files

• decile – list of percentiles

• outdir (str) – Output directory

create_outfile()

Create output data set.

Variables:

outdataset (utils.prosimu.prosimu) – output data set

decile

list of percentiles

deciles()

Calculates percentiles given in decile for variables in variables and adds them to the output data set including the corresponding dimension, coordinate variable and attributes.

init_outfile()

Copy global attributes, dimensions and standard variables from the first simulation file to the output file.

median()

Calculates the median for variables in variables and adds the median to the output data set including the corresponding attributes.

outfile

output filename

postprocess()

Do postprocessing

1. create output file

2. copy global attributes and standard variables to the output file

3. calculate deciles for the data variables and put them to the output file

4. close all input and output data sets.

Calls create_outfile(), init_outfile(), deciles() and close methods for all data sets.

property standardvars

variables always written to the output file

variables

list of variables

class plots.pearps2m.postprocess.EnsembleStation

Bases: Ensemble

Class for ensemble simulations at station locations.

Variables:

InfoMassifs – Information on the Massif the station is situated.

build_filename(station, alti)

Construct a filename from a station number

Parameters:

• station – station number

• alti – altitude level (not used)

Returns:

filename

Return type:

station number as 8digit integer.

build_title(station, alti)

Construct a title from a station number and altitude level.

Parameters:

• station – station number

• alti – station altitude

Returns:

title string composed of the station name and the altitude.

Return type:

unicode str

property geo

Geometry information.

Returns:

“stations”

get_metadata(**kwargs)

Construct filenames and plot titles from altitude and station information

Parameters:

kwargs –

Returns:

a list of filenames and a list of plot titles

get_station()

Read station numbers from the first simulation file.

Returns:

station numbers

Return type:

numpy array

class plots.pearps2m.postprocess._EnsembleMassif

Bases: Ensemble

Metaclass for ensemble simulations on a massif geometry (SAFRAN like).

Variables:

InfoMassifs – Information of Massifs

build_filename(massif, alti)

Construct a filename from massif and altitude information.

Parameters:

• massif – a massif number

• alti – an altitude level

Returns:

filename

Return type:

str

build_title(massif, alti)

Construct a figure title from massif and altitude information.

Parameters:

• massif – a massif number

• alti – an altitude level

Returns:

a title

Return type:

unicode str

property geo

Geometry

Returns:

“massifs”

get_massifdim()

Read massif_num variable from the first simulation file.

Returns:

massif numbers

Return type:

numpy array

get_massifvar()

Read “massif” variable from the first simulation file.

Returns:

massif numbers

Return type:

numpy array

get_metadata(nolevel=False)

Construct filenames and plot titles from massif and altitude variables in the first simulation file.

Parameters:

nolevel – if True the altitude is not included in the filenames and titles.

Returns:

a list of filenames and a list of titles

Return type:

two lists.

read(varname)

Read data for a given variable name into the ensemble instance variable.

Parameters:

varname (str) – variable name

class plots.pearps2m.postprocess.config

Bases: object

Configuration passed to S2MExtractor and to be used in vortex toolboxes.

1. checks and converts dates

2. creates output directories if they don’t exist.

__init__()

1. checks and converts dates

2. creates output directories if they don’t exist.

alternate_xpid = ['OPER@lafaysse']

Alternative experiment id

diroutput

output directory

diroutput_maps

output directory for maps

diroutput_plots

output directory for other plots

list_members = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35]

35 for determinstic member, 36 for sytron, 0-34 for PEARP members

previ = True

False for analysis, True for forecast

rundate

Date of model run bronx.stdtypes.date.Date

xpid = 'oper'

Operational chain

plots.pearps2m.postprocess.main(c)

Parameters:

c – config

plots.pearps2m.postprocess.parse_options(arguments)

Treat options passed to the script.

Parameters:

arguments – arguments from calling the script

Returns:

options