Vortex-cen HPC unit tasks

CEN research unit tasks for HPC.

oper_research_mixin.py

This modules defines specific CEN common addons for research and operational tasks. Multiple inheritance together with the standard Task class is required to use this module.

class vortex_cen.tasks.oper_research_mixin.CENTaskMixIn

Bases: object

Useful methods for any CEN (oper or research) task

property get_reprod_info

Provide information that should be stored in output files for reproducibility purposes

reforecast_filter_execution_error(exc)

Define the behaviour in case of errors in the reforecast task.

s2moper_filter_execution_error(exc)

Define the behaviour in case of errors.

For S2M chain, the errors do not raise exception if the deterministic run and if less than 5 members produce errors. Note than unavailability of members do not produce errors managed by effective_inputs), therefore more than 5 errors is a critical anomaly.

s2moper_report_execution_error(exc, **kw_infos)

Send error mail in case of fatal errors in the oper chain.

s2moper_report_execution_warning(exc, **kw_infos)

Send warning mail in case of non-fatal errors in the oper chain.

class vortex_cen.tasks.oper_research_mixin.CENTaskMixIn

Bases: object

Useful methods for any CEN (oper or research) task

property get_reprod_info

Provide information that should be stored in output files for reproducibility purposes

reforecast_filter_execution_error(exc)

Define the behaviour in case of errors in the reforecast task.

s2moper_filter_execution_error(exc)

Define the behaviour in case of errors.

For S2M chain, the errors do not raise exception if the deterministic run and if less than 5 members produce errors. Note than unavailability of members do not produce errors managed by effective_inputs), therefore more than 5 errors is a critical anomaly.

s2moper_report_execution_error(exc, **kw_infos)

Send error mail in case of fatal errors in the oper chain.

s2moper_report_execution_warning(exc, **kw_infos)

Send warning mail in case of non-fatal errors in the oper chain.

research_task_base.py

Main class used for all CEN HPC research tasks.

class vortex_cen.tasks.research_task_base._CenResearchTask(*args, **kw)

Bases: Task, CENTaskMixIn

Abstract class defining the common sequence of actions for CEN’s vortex tasks.

A vortex task is the sequence of actions to execute a single algo component.

It always follows this procedure (definied in the main process method) :

1. fetch all necessary input resources (files):

   • from an archive machine –> on a transfert node (‘early-fetch’)

   • from the local machine –> on a compute node (‘fetch’)

2. execute the algo component (an executable, a script or·a sequence of instructions)

   of inputs resources necessary to run an algo component (an executable, a script or a sequence of instructions) –> ‘compute’

3. save output resources (files produced or modified by the algo component)

   • to the local machine –> on a compute node (‘backup’)

   • to an archive machine –> on a compute node (‘late-backup’)

To implement a new task inheriting from this abstract class, implement a subset of the following methods:

• get_remote_inputs

• get_local_inputs

• algo

• launch_algo

• put_outputs

See their respective documentation for more details.

Doc: http://intra.cnrm.meteo.fr/algopy/trainings/vortex_dev2022_1/presentation/beamer/vortex_dev_jobs2_presentation.pdf

Initialise class attributes for dynamic documentation.

algo()

Implement this method to call your task’s algo component. This method should return a valid AlgoComponent object.

property debug

Enter ‘debug’ mode to preserve the working directory even after a succesfull execution.

Associated configuration variable:

• debug Enter ‘debug’ mode, default : False type debug: bool

defaults(extras)

Set toolbox defaults, extended with actual arguments extras.

Parameters:

extras (dict) – items to add to vortex.defaults.

diff()

Implement this method in your task to compare output with a reference file.

force_configuration_variables()

Implement this method to force the value of some configuration variables in specific use cases of the task. In particular, if an input for the task comes from the output of a previous task in the driver, the “block” of the input is imposed by the output block of the producing task.

get_forcing(localname='FORCING_[datebegin:ymdh]_[dateend:ymdh].nc', namespace='vortex.multi.fr', fatal=True)

Method to get meteorological forcing file(s) covering the simulation period. Look for files covering sub-periods defined by the io_duration configuration variable (current values: “monthly”, “yearly”, “full”)

Parameters:

• localname (str) – local footprint (how to name the file in the working directory). This is an algo/task-specific argument. Default name depends on the actual datebegin/dateend of each file. WARNING : in case a unique value is provided the user should ensure that a single file will be retrieved

• namespace (str) – namespace for fetching the forcing files. Default: vortex.multi.fr

Mandatory configuration variables:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

Optional configuration variables:

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str, values: “flat@cen”, “date@cen”, “date@std”

• forcing_intent intent footprint (local file permissions), default “in” Possible values : “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_date date footprint corresponding to the rundate of the forcing. Useful to get the forcing of the monthly reanalysis for example. default is forcing_dateend.

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values : ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values : “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

TODOprévoir un mécanisme pour rendre des déclarer les arguments obligatoires / optionnels pour

chaque tâche (ex: member)

get_list_dates(duration='yearly')

Get the list of datebegin/dateend corresponding to the different time periods covered by IO files from the actual simulation’s datebegin/dateend arguments.

Parameters:

duration (str) – Time period covered by individual files.

get_list_members()

Return the complete list of ensemble members from either - the exact ‘member’ value (int) –> returns [member] - the ‘members’ list (FPList or list) –> returns the list of members - the number of members ‘nmembers’ (int) –> returns the list of ‘nmembers’ values starting from 1

get_local_inputs()

Implement this method in your task to fetch all resources already stored on the local (HPC) cache from a compute node.

get_remote_inputs()

Implement this method in your task to fetch all resources stored remotely (on Hendrix, sxcen,…) from a transfer node.

launch_MPI_executable(algo, mpiopts=None)

Run executable with MPI.

Parameters:

• algo – AlgoComponent object

• mpiopts – dict with MPI options nnodes=…, nprocs=…, ntasks=…

launch_algo(algo, **kw)

Implement this method in your task’s algo component. The implementation should define how to run the algo component, or call one of the standard methods: launch_MPI_executable(), launch_python_algo()

Parameters:

• algo – AlgoComponent object

• kw

launch_executable(algo)

run executable without MPI.

Parameters:

algo – AlgoComponent object

launch_python_algo(algo, **kw)

Run your task’s algo component. For algo components consisting of python code. :param algo: AlgoComponent object :param kw: keyword arguments dict

property namespace_out

Namespace for output toolboxes

preprocess()

Pre-processing step to set usefull class variables.

Associated (optional) configuration variables:

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. It is used to retrieve the list of datebegin and dateend footprints for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type io_duration: str

process()

Main method definig the task’s sequence of actions

put_outputs()

Implement this method in your task to save resources remotely (on Hendrix, sxcen,…) from a transfer node.

unittest()

Implement this method in unittest tasks to monitor the test results.

update_attributes(mandatory, optional, overwrite=None)

Update class attributes for dynamic documentation

configuration_variables.py

Documentation of most frequently used configuration variables. See Dynamic documentation of unit tasks for more information on the documentation syntax conventions.

surfex

Tasks allowing to launch any SURFEX executable.

commons.py

MixIn input common to all SURFEX tasks.

class vortex_cen.tasks.surfex.commons.SurfexCommonsMixin

Bases: object

Mixin class that provides methods to get common SURFEX inputs.

get_drdt_bst_fit()

Get drdt_bst_fit_60.nc from uenv Crocus metamorphism parameters mandatory to run OFFLINE, PREP or PGD

Configuration Variables used:

• consts_surfex_uenv or if not present uenv User Environment in which the following resources are to be retrieved : - ecoclimapI_covers_param.bin - ecoclimapII_eu_covers_param.bin

  Format : uenv:{uenv_name}@{user}

get_ecoclimap()

Get ecoclimapI_covers_param.bin, ecoclimapII_eu_covers_param.bin, Binary ECOCLIMAP I files are mandatory to run OFFLINE and taken from the uenv Binary ECOCLIMAP II files are mandatory to run OFFLINE and taken from the uenv

Configuration Variables used:

• surfex_uenv or if not present uenv User Environment in which the following resources are to be retrieved : - ecoclimapI_covers_param.bin - ecoclimapII_eu_covers_param.bin

  Format : uenv:{uenv_name}@{user}

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

get_init_TG_from_cache_or_archive(fatal=True, cache_only=False)

Method to get an init_tg from the vortex cache or the archive.

param fatal:

If True, the method raises a fatal error if the file could not be fetched. Default: True. Should be False only in tasks that implement a second option for fetching an init_tg file, for example from an uenv.

type fatal:

bool

param cache_only:

If True, the method gets the file from the cache only. Default: False. cache_only=False should be used in tasks that are supposed to fetch an init_tg file as a remote input, or check if such a file already exists for the given experiment. cache_only=True should be used in tasks where the init_tg file is considered a local input, that means that it has been either fetched or produced by a previous task in the driver.

type cache_only:

bool

return:

init_tg toolbox

Configuration Variables used:

• tg_xpid or xpid experiment id the init_TG.nc file should be fetched from.

• tg_user name of the user that produced the target the init_TG.nc file. Default: None

• tg_geometry or geometry geometry of the init_TG. Logically the same as for the rest of the simulation

• tg_vapp or vapp Application name to search the init_TG.nc file.

• tg_vconf or vconf Configuration name to search the init_TG.nc file.

get_init_TG_from_uenv(fatal=True)

Fetch an init_TG.nc file from an uenv

Parameters:

fatal – If True, the method raises a fatal error if the file could not be fetched. Default: True. Should be False only in tasks that implement a second option for fetching an init_tg file, for example from the cache or archive.

Returns:

init_tg toolbox

Configuration Variables used:

• tg_geometry or geometry geometry of the init_TG. Logically the same as for the rest of the simulation

• surfex_uenv or if not present uenv User Environment from which the init_TG.nc file should be fetched.

  Format : uenv:{uenv_name}@{user}

• tg_gvar key to look up the init_TG.nc file in the uenv the file should come from. Default: climtg_[geometry::area]

get_namelist()

Get the OPTIONS.nam namelist. If an allow_path=True AND a namelist_path configuration variables are provided, fetch the user-provided target file (WARNING : non-reproducible case). In any other case, fetch the namelist from a uenv.

get_pgd_file_from_cache()

In the general research case, the PGD comes from the vortex cache. For “stable” configurations such as the reanalysis, it comes from a UEnv/GEnv.

Configuration Variables used:

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID. defaults to xpid. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp. defaults to vapp. type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf. defaults to vconf. type: str

• pgd_vortex1 True if the pgd file was produced with vortex1 and uses vortex1 naming conventions. default: False type: bool

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

get_pgd_file_from_cache_or_archive(fatal=True)

Method to be used in tasks that fetch or try to fetch the pgd file from the cache or archive.

Parameters:

fatal (bool) – If True, the method raises a fatal error if the file could not be fetched. Default: True. Should be False only in tasks that implement a second option for fetching a pgd file, for example from an uenv.

Returns:

pgd toolbox

Configuration Variables used:

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID. defaults to xpid. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp. defaults to vapp. type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf. defaults to vconf. type: str

• pgd_vortex1 True if the pgd file was produced with vortex1 and uses vortex1 naming conventions. default: False type: bool

• pgd_user Name of the user who produced the PGD file. Default: None. type: str

• pgd_geometry geometry of the pgd file. Default: geometry. type: str, footprints.stdtypes.FPList

get_pgd_file_from_uenv(fatal=True)

Get PGD.nc file from UEnv

Parameters:

fatal – If True, the method raises a fatal error if the file could not be fetched. Default: True. Should be False only in tasks that implement a second option for fetching a pgd file, for example one archived from a previous experiment.

Returns:

pgd toolbox

Configuration Variables used:

• surfex_uenv or if not present uenv User Environment from which the PGD.nc file should be fetched.

  Format : uenv:{uenv_name}@{user}

• pgdnc_gvar variable name of the pgd file in the uenv. Default: pgd_[geometry::tag].

• pgd_geometry geometry of the pgd file. Default: geometry. type: str, footprints.stdtypes.FPList

get_prep_file_from_cache_or_archive(fatal=True, cache_only=False, local='PREP.nc')

Standard method to get a PREP file from the cache or archive. The PREP.nc file contains the initial conditions of the snowpack at the beginning of the simulation.

Parameters:

• fatal (bool) – If True, the method raises a fatal error if the file could not be fetched. Default: True. Should be False only in tasks that implement a second option for fetching a pgd file, for example from an uenv.

• cache_only (bool) – If True, the method gets the PREP file from the cache only. Default: False. cache_only=False should be used in tasks that are supposed to fetch a PREP file as a remote input, or check if such a file already exists for the given experiment. cache_only=True should be used in tasks where the prep file is considered a local input, that means that it has been either fetched or produced by a previous task in the driver.

Returns:

a toolbox input with the PREP file.

Configuration Variables used:

• prep_xpid or xpid Experiment id the prep file should be searched for or put in cache.

• prep_user name of the user who produced the PREP file. Default: None.

• prep_datevalidity or datebegin Validity date of the prep file. Default is datebegin.

• prep_vapp or vapp Application name to search the PREP.nc file.

• prep_vconf or vconf Configuration name to search the PREP.nc file.

• prep_vortex1 type: bool. True if the requested PREP.nc file was produced with vortex 1 and thus uses vortex 1 naming conventions. Default is False.

• prep_geometry or geometry geometry of the PREP file.

• prep_namebuild Default: flat@cen

• prep_block block part of the data tree to search for the PREP.nc file. Default is prep.

• prep_member or member If the PREP.nc file comes from an ensemble, a member can be chosen. Default is None.

• prep_cutoff Can be used to select a PREP file coming from an operational forecast (forecast) or

  analysis (assimilation). Default is None. Might be useful for reforecasts.

offline.py

Tasks designed to launch the OFFLINE executable with MPI parallelisation.

class vortex_cen.tasks.surfex.offline.OfflineCommonsMixin

Bases: SurfexCommonsMixin

Common OFFLINE-specific IO resources.

get_executable_from_path(fatal=True)

Get OFFLINE executable locally

Parameters:

fatal (bool) – If True, fails if the executable was not found. Default: True

Configuration variables used:

• exesurfex absolute path to the OFFLINE executable type: str

get_executable_from_uenv(mpi=True, fatal=True)

Get OFFLINE executable from Uenv

Parameters:

• mpi (bool) – If True, fetch offline executable with MPI support named master_offline_mpi, if False, fetch offline executable without MPI support named master_offline_nompi. The names can be overwritten using the offline_gvar configuration variable. Default: True

• fatal (bool) – If True, fails if the executable was not found. Default: True

Configuration variables used:

• surfex_uenv or uenv

• offline_gvar or one of master_offline_mpi or master_offline_nompi depending on the value of the mpi parameter.

class vortex_cen.tasks.surfex.offline._Offline(*args, **kw)

Bases: OfflineCommonsMixin, _CenResearchTask

Task: _Offline

Abstract task for OFFLINE binary execution.

SURFEX/OFFLINE documentation : https://umr-cnrm.github.io/snowtools-doc/misc/surfex.html

Inputs:

• FORCING.nc files(s) (near-surface meteorological conditions during the simulation period)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography)

• PREP.nc (initial conditions)

Outputs:

• PRO.nc Snowpack simulations covering the entire simulation period

• PREP.nc SURFEX/Crocus model state variables at the end of the simulation

• CUMUL.nc TODO Compléter et CHECKER la doc

• DIAG.nc TODO Compléter et CHECKER la doc

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid User-defined Experiment identifier (WARNING : 4-digit strings prohibited) type: str

• surfex_uenv or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • drdt_bst_fit_60.nc

  • OFFLINE executable

  Format : uenv:{uenv_name}@{user} type: str

• nprocs Number of process to allocate to the execution of the MPI binary type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary type: int

Optional configuration variables (other than forcing-specific ones):

• exesurfex Path to the executable if it should come from a local path.

• offline_gvar specify the name of the offline executable in the uenv. Default is master_offline_mpi if the mpi parameter is True and master_offline_nompi otherwise.

• member Simulation member. NB : This is a deterministic task, only one single member value can be provided type: int

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user User who produced the target PGD file. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• prep_xpid Experiment Identifier of the PREP file, if different from the task’s XPID type: str

• prep_user User who produced the target PREP file. type: str

• prep_member Member associated to the PREP file if it comes from an ensemble (after a SODA run) NB : This is a deterministic task, only one single member value can be provided type: int

• prep_vapp vapp of the PREP file, if different from the task’s vapp type: str

• prep_vconf vconf of the PREP file, if different from the task’s vconf type: str

• prep_datevalidity Validity date of the PREP file (if different from datebegin) type: str

• prep_block block of the PREP file (default ‘prep’, but can be different after an assimilation step) type: str

• prep_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type: bool

• august_threshold Threshold to apply to the snow water equivalent (in kg/m2) each 1st August (default: -999) type: int

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• drhook Activate / deactivate the profiling with DRHOOK (default: False) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• nnodes Number of available nodes for MPI parallelisation type: int

• nprocs Number of available processors for MPI parallelisation type: int

• ntasks Number of MPI tasks type: int

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• diff_block block of the reference file used for reproducibility test

Forcing related configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.offline.OfflineMpi(*args, **kw)

Bases: _Offline

Task: OfflineMpi

Task for the execution of OFFLINE binary with MPI parallelisation.

Inputs:

• FORCING.nc files(s) (near-surface meteorological conditions during the simulation period)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography)

• PREP.nc (initial conditions)

Outputs:

• PRO.nc Snowpack simulations covering the entire simulation period

• PREP.nc SURFEX/Crocus model state variables at the end of the simulation

• CUMUL.nc TODO Compléter et CHECKER la doc

• DIAG.nc TODO Compléter et CHECKER la doc

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid User-defined Experiment identifier (WARNING : 4-digit strings prohibited) type: str

• surfex_uenv or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • drdt_bst_fit_60.nc

  • OFFLINE executable

  Format : uenv:{uenv_name}@{user} type: str

• nprocs Number of process to allocate to the execution of the MPI binary type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary type: int

Optional configuration variables (other than forcing-specific ones):

• exesurfex Path to the executable if it should come from a local path.

• offline_gvar specify the name of the offline executable in the uenv. Default is master_offline_mpi if the mpi parameter is True and master_offline_nompi otherwise.

• member Simulation member. NB : This is a deterministic task, only one single member value can be provided type: int

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user User who produced the target PGD file. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• prep_xpid Experiment Identifier of the PREP file, if different from the task’s XPID type: str

• prep_user User who produced the target PREP file. type: str

• prep_member Member associated to the PREP file if it comes from an ensemble (after a SODA run) NB : This is a deterministic task, only one single member value can be provided type: int

• prep_vapp vapp of the PREP file, if different from the task’s vapp type: str

• prep_vconf vconf of the PREP file, if different from the task’s vconf type: str

• prep_datevalidity Validity date of the PREP file (if different from datebegin) type: str

• prep_block block of the PREP file (default ‘prep’, but can be different after an assimilation step) type: str

• prep_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type: bool

• august_threshold Threshold to apply to the snow water equivalent (in kg/m2) each 1st August (default: -999) type: int

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• drhook Activate / deactivate the profiling with DRHOOK (default: False) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• nnodes Number of available nodes for MPI parallelisation type: int

• nprocs Number of available processors for MPI parallelisation type: int

• ntasks Number of MPI tasks type: int

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• diff_block block of the reference file used for reproducibility test

Forcing related configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.offline._Offline_NOMPI(*args, **kw)

Bases: _Offline

Task : _Offline_NOMPI

Abstract task for the execution of OFFLINE binary without MPI parallelisation.

Inputs:

• FORCING.nc files(s) (near-surface meteorological conditions during the simulation period)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography)

• PREP.nc (initial conditions)

Outputs:

• PRO.nc Snowpack simulations covering the entire simulation period

• PREP.nc SURFEX/Crocus model state variables at the end of the simulation

• CUMUL.nc TODO Compléter et CHECKER la doc

• DIAG.nc TODO Compléter et CHECKER la doc

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid User-defined Experiment identifier (WARNING : 4-digit strings prohibited) type: str

• surfex_uenv or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • drdt_bst_fit_60.nc

  • OFFLINE executable

  Format : uenv:{uenv_name}@{user} type: str

• nprocs Number of process to allocate to the execution of the MPI binary type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary type: int

Optional configuration variables (other than forcing-specific ones):

• exesurfex Path to the executable if it should come from a local path.

• offline_gvar specify the name of the offline executable in the uenv. Default is master_offline_mpi if the mpi parameter is True and master_offline_nompi otherwise.

• member Simulation member. NB : This is a deterministic task, only one single member value can be provided type: int

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user User who produced the target PGD file. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• prep_xpid Experiment Identifier of the PREP file, if different from the task’s XPID type: str

• prep_user User who produced the target PREP file. type: str

• prep_member Member associated to the PREP file if it comes from an ensemble (after a SODA run) NB : This is a deterministic task, only one single member value can be provided type: int

• prep_vapp vapp of the PREP file, if different from the task’s vapp type: str

• prep_vconf vconf of the PREP file, if different from the task’s vconf type: str

• prep_datevalidity Validity date of the PREP file (if different from datebegin) type: str

• prep_block block of the PREP file (default ‘prep’, but can be different after an assimilation step) type: str

• prep_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type: bool

• august_threshold Threshold to apply to the snow water equivalent (in kg/m2) each 1st August (default: -999) type: int

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• drhook Activate / deactivate the profiling with DRHOOK (default: False) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• diff_block block of the reference file used for reproducibility test

Forcing related configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.offline.Offline_Mpi_Uenv(*args, **kw)

Bases: OfflineMpi

Task: Offline_MPI_Uenv

Get OFFLINE executable from a User Environment.

NB: This is the task to use to guarantee the simulation’s reproductibility

class vortex_cen.tasks.surfex.offline.OfflineMpiDailyPrep(*args, **kw)

Bases: OfflineMpi

Do a surfex simulation with daily prep file output.

Inputs:

• FORCING.nc files(s) (near-surface meteorological conditions during the simulation period)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography)

• PREP.nc (initial conditions)

Outputs:

• PRO.nc Snowpack simulations covering the entire simulation period

• PREP.nc SURFEX/Crocus model state variables at the end of the simulation

• CUMUL.nc TODO Compléter et CHECKER la doc

• DIAG.nc TODO Compléter et CHECKER la doc

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid User-defined Experiment identifier (WARNING : 4-digit strings prohibited) type: str

• surfex_uenv or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • drdt_bst_fit_60.nc

  • OFFLINE executable

  Format : uenv:{uenv_name}@{user} type: str

• nprocs Number of process to allocate to the execution of the MPI binary type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary type: int

Optional configuration variables (other than forcing-specific ones):

• offline_gvar specify the name of the offline executable in the uenv. Default is master_offline_mpi if the mpi parameter is True and master_offline_nompi otherwise.

• member Simulation member. NB : This is a deterministic task, only one single member value can be provided type: int

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user User who produced the target PGD file. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• prep_xpid Experiment Identifier of the PREP file, if different from the task’s XPID type: str

• prep_user User who produced the target PREP file. type: str

• prep_member Member associated to the PREP file if it comes from an ensemble (after a SODA run) NB : This is a deterministic task, only one single member value can be provided type: int

• prep_vapp vapp of the PREP file, if different from the task’s vapp type: str

• prep_vconf vconf of the PREP file, if different from the task’s vconf type: str

• prep_datevalidity Validity date of the PREP file (if different from datebegin) type: str

• prep_block block of the PREP file (default ‘prep’, but can be different after an assimilation step) type: str

• prep_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type: bool

• august_threshold Threshold to apply to the snow water equivalent (in kg/m2) each 1st August (default: -999) type: int

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• drhook Activate / deactivate the profiling with DRHOOK (default: False) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• nnodes Number of available nodes for MPI parallelisation type: int

• nprocs Number of available processors for MPI parallelisation type: int

• ntasks Number of MPI tasks type: int

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• diff_block block of the reference file used for reproducibility test

Forcing related configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.offline.OfflineAssim(*args, **kw)

Bases: OfflineOpenloop

This is the task for an OFFLINE-MPI execution after a snow data assimilation step. Each simulaiton member is initialised by a different PREP file identified by its member value.

class vortex_cen.tasks.surfex.offline.OfflineOpenloop(*args, **kw)

Bases: OfflineMpi

This is the task for an OFFLINE-MPI execution before any data assimilation. All members are initialised by the same PREP file, not associated to any member.

class vortex_cen.tasks.surfex.offline.OfflineLocalForcing(*args, **kw)

Bases: OfflineMpi

Task : OfflineLocalForcing

SURFEX/OFFLINE documentation : https://umr-cnrm.github.io/snowtools-doc/misc/surfex.html

Inputs:

• FORCING.nc files(s) (near-surface meteorological conditions during the simulation period)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from the execution of the “PreProcess”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography) retrieved or produced by the GetPgd1D task

• PREP.nc (initial conditions) retrieved or produced by the GetPrep task

Outputs:

• PRO.nc Snowpack simulations covering the entire simulation period

• PREP.nc SURFEX/Crocus model state variables at the end of the simulation

offline_ensemble.py

Tasks designed to launch an OFFLINE executable WITHOUT MPI parallelisation several time in parallel.

class vortex_cen.tasks.surfex.offline_ensemble.Escroc(*args, **kw)

Bases: _Offline

Task : Escroc

Multiple executions of an OFFLINE binary with a single meteorological FORCING but different Crocus physics (namelists) and no MPI parallelization.

Lafaysse et al. (2017) : https://tc.copernicus.org/articles/11/1173/2017/

Inputs:

• FORCING.nc files(s) (near-surface meteorological conditions during the simulation period)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography)

• PREP.nc (initial conditions)

Outputs:

• PRO.nc Snowpack simulations covering the entire simulation period

• PREP.nc SURFEX/Crocus model state variables at the end of the simulation

• CUMUL.nc TODO Compléter et CHECKER la doc

• DIAG.nc TODO Compléter et CHECKER la doc

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid User-defined Experiment identifier (WARNING : 4-digit strings prohibited) type: str

• surfex_uenv or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • drdt_bst_fit_60.nc

  • OFFLINE executable

  Format : uenv:{uenv_name}@{user} type: str

• nmembers number of ensemble members.

Optional configuration variables (other than forcing-specific ones):

• exesurfex Path to the executable if it should come from a local path.

• offline_gvar specify the name of the offline executable in the uenv. Default is master_offline_mpi if the mpi parameter is True and master_offline_nompi otherwise.

• member Simulation member. NB : This is a deterministic task, only one single member value can be provided type: int

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user User who produced the target PGD file. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• prep_xpid Experiment Identifier of the PREP file, if different from the task’s XPID type: str

• prep_user User who produced the target PREP file. type: str

• prep_member Member associated to the PREP file if it comes from an ensemble (after a SODA run) NB : This is a deterministic task, only one single member value can be provided type: int

• prep_vapp vapp of the PREP file, if different from the task’s vapp type: str

• prep_vconf vconf of the PREP file, if different from the task’s vconf type: str

• prep_datevalidity Validity date of the PREP file (if different from datebegin) type: str

• prep_block block of the PREP file (default ‘prep’, but can be different after an assimilation step) type: str

• prep_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type: bool

• august_threshold Threshold to apply to the snow water equivalent (in kg/m2) each 1st August (default: -999) type: int

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• drhook Activate / deactivate the profiling with DRHOOK (default: False) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• nnodes Number of available nodes for MPI parallelisation type: int

• nprocs Number of available processors for MPI parallelisation type: int

• ntasks Number of MPI tasks type: int

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• diff_block block of the reference file used for reproducibility test

• subensemble Name of the predefined escroc sub-ensemble to use. typ: str, default: “E2”

• output_storage Name of the archive/server where the output files will be stored. type: str

Forcing related configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.offline_ensemble.CrocO(*args, **kw)

Bases: Escroc

Task : CrocO

Multiple executions of an OFFLINE binary with an ensemble of FORCING files and potentially different Crocus physics (namelists).

This task is strongly linked to the croco driver (vortex_cen/Crocus/assim/drivers/croco.py).

In particular, some configuration variables derived from the list of assimilation dates (configuration variable “assimdates” provided in the configuration file by the user) come from the internal loop within the croco driver : * assimdate_prev refers to the assimilation date of the last interation * assimdate (singular !) refers to the assimilation date of the current iteration * assimdate_next refers to the assimilation date of the next iteration

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid User-defined Experiment identifier (WARNING : 4-digit strings prohibited) type: str

• surfex_uenv or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • drdt_bst_fit_60.nc

  • OFFLINE executable

  Format : uenv:{uenv_name}@{user} type: str

• nmembers number of ensemble members.

Optional configuration variables (other than forcing-specific ones):

• exesurfex Path to the executable if it should come from a local path.

• offline_gvar specify the name of the offline executable in the uenv. Default is master_offline_mpi if the mpi parameter is True and master_offline_nompi otherwise.

• member Simulation member. NB : This is a deterministic task, only one single member value can be provided type: int

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user User who produced the target PGD file. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• prep_xpid Experiment Identifier of the PREP file, if different from the task’s XPID type: str

• prep_user User who produced the target PREP file. type: str

• prep_member Member associated to the PREP file if it comes from an ensemble (after a SODA run) NB : This is a deterministic task, only one single member value can be provided type: int

• prep_vapp vapp of the PREP file, if different from the task’s vapp type: str

• prep_vconf vconf of the PREP file, if different from the task’s vconf type: str

• prep_datevalidity Validity date of the PREP file (if different from datebegin) type: str

• prep_block block of the PREP file (default ‘prep’, but can be different after an assimilation step) type: str

• prep_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type: bool

• august_threshold Threshold to apply to the snow water equivalent (in kg/m2) each 1st August (default: -999) type: int

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• drhook Activate / deactivate the profiling with DRHOOK (default: False) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• nnodes Number of available nodes for MPI parallelisation type: int

• nprocs Number of available processors for MPI parallelisation type: int

• ntasks Number of MPI tasks type: int

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• diff_block block of the reference file used for reproducibility test

• subensemble Name of the predefined escroc sub-ensemble to use. typ: str, default: “E2”

• output_storage Name of the archive/server where the output files will be stored. type: str

Forcing related configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.offline_ensemble.EscrocResearch(*args, **kw)

Bases: Escroc

Task: EscrocResearch

SURFEX/OFFLINE documentation : https://umr-cnrm.github.io/snowtools-doc/misc/surfex.html

Inputs:

• FORCING.nc files(s) (near-surface meteorological conditions during the simulation period)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from the execution of the “PreProcess”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography) retrieved or produced by the GetPgd1D task

• PREP.nc (initial conditions) retrieved or produced by the GetPrep task

Outputs:

• PRO.nc Snowpack simulations covering the entire simulation period

• PREP.nc SURFEX/Crocus model state variables at the end of the simulation

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid User-defined Experiment identifier (WARNING : 4-digit strings prohibited) type: str

• surfex_uenv or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • drdt_bst_fit_60.nc

  • OFFLINE executable

  Format : uenv:{uenv_name}@{user} type: str

• nmembers number of ensemble members.

Optional configuration variables (other than forcing-specific ones):

• exesurfex Path to the executable if it should come from a local path.

• offline_gvar specify the name of the offline executable in the uenv. Default is master_offline_mpi if the mpi parameter is True and master_offline_nompi otherwise.

• member Simulation member. NB : This is a deterministic task, only one single member value can be provided type: int

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user User who produced the target PGD file. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• prep_xpid Experiment Identifier of the PREP file, if different from the task’s XPID type: str

• prep_user User who produced the target PREP file. type: str

• prep_member Member associated to the PREP file if it comes from an ensemble (after a SODA run) NB : This is a deterministic task, only one single member value can be provided type: int

• prep_vapp vapp of the PREP file, if different from the task’s vapp type: str

• prep_vconf vconf of the PREP file, if different from the task’s vconf type: str

• prep_datevalidity Validity date of the PREP file (if different from datebegin) type: str

• prep_block block of the PREP file (default ‘prep’, but can be different after an assimilation step) type: str

• prep_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type: bool

• august_threshold Threshold to apply to the snow water equivalent (in kg/m2) each 1st August (default: -999) type: int

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• drhook Activate / deactivate the profiling with DRHOOK (default: False) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• nnodes Number of available nodes for MPI parallelisation type: int

• nprocs Number of available processors for MPI parallelisation type: int

• ntasks Number of MPI tasks type: int

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for IO covering sub-periods. Possible values : “yearly”, “monthly” or “full” type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• diff_block block of the reference file used for reproducibility test

• subensemble Name of the predefined escroc sub-ensemble to use. typ: str, default: “E2”

• output_storage Name of the archive/server where the output files will be stored. type: str

Forcing related configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

pre_process.py

Tasks designed to launch the pre-process the OPTIONS.nam namelist before any SURFEX binary execution.

class vortex_cen.tasks.surfex.pre_process._Preprocess(*args, **kw)

Bases: SurfexCommonsMixin, _CenResearchTask

Task: _Preprocess

Abstract task for pre-processing namelist: add infos like points and dates from forcing to namelist.

Inputs:

• SURFEX namelist (OPTIONS.nam)

• FORCING file

Outputs:

• Modified and ready-to-use SURFEX namelist

Mandatory configuration variables:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

Optional configuration variables:

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.pre_process.PreprocessNamelist(*args, **kw)

Bases: _Preprocess

Task: PreprocessNamelist

Task for pre-processing a namelist defined by its absolute path or uenv NB: This is the task to use to guarantee the simulation’s reproductibility

Mandatory configuration variables:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

Optional configuration variables:

• namelist_path absolute path to the namelist file if the namelist is not in the uenv.

• consts_surfex_uenv or if not present uenv User Environment from which the namelist file should be fetched. Format : uenv:{uenv_name}@{user}

• namelist_source In an UEnv, several namelistes can be present in an .tar archive, the source footprint allows to define the exact name of the nameliste to fetch. For example, OPTIONS_default.nam.

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.pre_process.SodaNamelistPreprocess(*args, **kw)

Bases: SurfexCommonsMixin, _CenResearchTask

Task: SodaNamelistPreprocess

Pre-process SURFEX namelist for SODA executable

Inputs:

• SODA namelist (OPTIONS.nam)

Outputs:

• SODA namelist (OPTIONS.nam)

Mandatory configuration variables

• uenv User Environment in which the namelist is to be retrieved type: str

• namelist_source Name of the namelist in the user environment type: str

• nmembers Number of ensemble members in the background state type: int

• xpid Experiment Identifier type: str

prep.py

Tasks designed to launch the PREP executable.

class vortex_cen.tasks.surfex.prep.PrepCommonsMixin

Bases: SurfexCommonsMixin

Mixin class prviding methods for fetching a PREP executable.

Optional configuration variables:

• surfex_uenv If the executable should come from an uenv. Default is uenv

• prep_gvar specify the name of the PREP executable in the uenv. Default is master_prep_mpi if the mpi parameter is True and master_prep_nompi otherwise.

• exesurfex Path to the executable if it should come from a local path.

get_prep_exe_from_path()

Fetch a PREP executable from a local path.

Configuration variables used:

• exesurfex Path to the executable if it should come from a local path.

get_prep_exe_from_uenv(mpi=True, fatal=True)

method to get a PREP executable from uenv.

Parameters:

• mpi – True if an executable with MPI support should be fetched, False otherwise. Default is True.

• fatal – True if failing to fetch the executable should cause a fatal error, False otherwise. Default is True.

Configuration variables used:

• surfex_uenv If the executable should come from an uenv. Default is uenv

• prep_gvar specify the name of the PREP executable in the uenv. Default is master_prep_mpi if the mpi parameter is True and master_prep_nompi otherwise.

class vortex_cen.tasks.surfex.prep._PrepConstruct(*args, **kw)

Bases: PrepCommonsMixin, _CenResearchTask

Task: _PrepConstruct

Abstract task for the generation of initial conditions (PREP.nc file)

Inputs:

• FORCING file (used only to set the simulation’s geometry in the namelist)

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• Init_TG.nc Initial values of ground temperature coming from the cache (put there by an execution of an InitClimGroundTemperature or GetClimGroundTemperature task)

• PGD.nc Ground physiography coming from the cache (put there by an execution of a Pgd* task or GetPgd1D task

Outputs:

• PREP.nc (initial conditions)

Mandatory configuration variables:

• date Date of validity of the PREP.nc file to generate. Default is datebegin type: str, Date

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• surfex_uenv or if not present uenv User Environment in which the following resources are to be retrieved : - ecoclimapI_covers_param.bin - ecoclimapII_eu_covers_param.bin - drdt_bst_fit_60.nc - PREP executable

  Format : uenv:{uenv_name}@{user}

• nprocs Number of process to allocate to the execution of the MPI binary type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary type: int

Optional configuration variables:

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user Name of the user who produced the PGD file type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

class vortex_cen.tasks.surfex.prep.FetchPrepFileOrMake(*args, **kw)

Bases: _PrepConstruct

Task: FetchPrepFileOrMake

Generation of initial conditions (PREP.nc file) Look if the requested PREP.nc file is available in the cache or archive. If not, calculate it. The output is put to the cache but not archived! Use the MakePrepFile class to generate and archive a PREP.nc file

WARNING : The simulation’s reproducibility can not be guaranteed with this task!

Inputs:

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• Init_TG.nc Initial values of ground temperature coming from the cache (put there by an execution of an InitClimGroundTemperature or GetClimGroundTemperature task)

• PGD.nc Ground physiography

Outputs:

• PREP.nc (initial conditions)

Mandatory configuration variables:

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• surfex_uenv or if not present uenv User Environment in which the following resources are to be retrieved : - ecoclimapI_covers_param.bin - ecoclimapII_eu_covers_param.bin - drdt_bst_fit_60.nc - PREP executable

  Format : uenv:{uenv_name}@{user}

Optional configuration variables:

• exesurfex Path to the executable if it should come from a local path.

• prep_gvar specify the name of the PREP executable in the uenv.

• prep_xpid or xpid Experiment id the prep file should be searched for or put in cache.

• prep_user name of the user who produced the PREP file. Default: None.

• prep_datevalidity Validity date of the prep file.

• prep_vapp or vapp Application name to search the PREP.nc file.

• prep_vconf or vconf Configuration name to search the PREP.nc file.

• prep_vortex1 type: bool. True if the requested PREP.nc file was produced with vortex 1 and thus uses vortex 1 naming conventions. Default is False.

• prep_geometry or geometry geometry of the PREP file.

• prep_namebuild Default: flat@cen

• prep_block block part of the data tree to search for the PREP.nc file. Default is prep.

• prep_member or member If the PREP.nc file comes from an ensemble, a member can be chosen. Default is None.

• prep_cutoff Can be used to select a PREP file coming from an operational forecast (forecast) or analysis (assimilation). Default is None. Might be useful for reforecasts.

class vortex_cen.tasks.surfex.prep.FetchPrepFileOrCrash(*args, **kw)

Bases: FetchPrepFileOrMake

Fetch a prep file from the archive (or vortex cache) and put it in the cache of the current experiment.

Mandatory configuration variables:

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• prep_xpid or xpid Experiment id the prep file should be searched for or put in cache.

• prep_user name of the user who produced the PREP file. Default: None.

• prep_datevalidity Validity date of the prep file.

• prep_vapp or vapp Application name to search the PREP.nc file.

• prep_vconf or vconf Configuration name to search the PREP.nc file.

• prep_vortex1 type: bool. True if the requested PREP.nc file was produced with vortex 1 and thus uses vortex 1 naming conventions. Default is False.

• prep_geometry or geometry geometry of the PREP file.

• prep_namebuild Default: flat@cen

• prep_block block part of the data tree to search for the PREP.nc file. Default is prep.

• prep_member or member If the PREP.nc file comes from an ensemble, a member can be chosen. Default is None.

• prep_cutoff Can be used to select a PREP file coming from an operational forecast (forecast) or analysis (assimilation). Default is None. Might be useful for reforecasts.

class vortex_cen.tasks.surfex.prep.PrepRefill(*args, **kw)

Bases: FetchPrepFileOrCrash

Task : PrepRefill

Refill a cache with a PREP file from another experiment.

Input:

• A single PREP.nc

Output:

• A copy of the input PREP.nc file into the cache of all members of the real-time chain

Mandatory configuration variables:

• geometry geometry of the forcing file(s)

• xpid Experiment identifier. type: str

• prep_datevalidity Validity date of the prep file.

• prep_xpid Experiment id the prep file should be searched for.

• rundate Date of run. choices: YYYYMMDD[03 06 09 12], type: str or Date

• datevalidity Date of validity of the PREP.nc file to generate. Default is datebegin type: str, Date

• members

Optional configuration variables:

• prep_user name of the user who produced the PREP file. Default: None.

• prep_vapp or vapp Application name to search the PREP.nc file.

• prep_vconf or vconf Configuration name to search the PREP.nc file.

• prep_vortex1 type: bool. True if the requested PREP.nc file was produced with vortex 1 and thus uses vortex 1 naming conventions. Default is False.

• prep_block block part of the data tree to search for the PREP.nc file. Default is prep.

• cutoff Target cutoff (refill an analysis or a forecast output). type: str, choices: ‘assimilation’, ‘production’, default: ‘assimilation’

class vortex_cen.tasks.surfex.prep.MakePrepFile(*args, **kw)

Bases: _PrepConstruct

Task : MakePrepFile

Task for the generation of initial conditions (PREP.nc file). By default the resulting file is archived. Use the namespace_out configuration variable to change this behavior.

Inputs:

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• Init_TG.nc Initial values of ground temperature coming from the cache (put there by an execution of an InitClimGroundTemperature or GetClimGroundTemperature task)

• PGD.nc Ground physiography coming from the cache (put there by an execution of a Pgd* task or GetPgd1D task

Outputs:

• PREP.nc (initial conditions)

Mandatory configuration variables:

• datevalidity Date of validity of the PREP.nc file to generate. Default is datebegin type: str, Date

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• surfex_uenv or if not present uenv User Environment in which the following resources are to be retrieved : - ecoclimapI_covers_param.bin - ecoclimapII_eu_covers_param.bin - drdt_bst_fit_60.nc - PREP executable

  Format : uenv:{uenv_name}@{user}

• nprocs Number of process to allocate to the execution of the MPI binary type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary type: int

Optionnal configuration variables:

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID type: str

• pgd_user Name of the user who produced the PGD file type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf type: str

• dailyprep TODO :comprendre avec Matthieu L les cas d’usages avec “dailyprep” (reforecast ?) type: bool

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

• exesurfex Path to the executable if it should come from a local path. Otherwise,

• surfex_uenv If the executable should come from an uenv. Default is uenv

• prep_gvar specify the name of the PREP executable in the uenv. Default is master_prep_mpi if the mpi parameter is True and master_prep_nompi otherwise.

pdg.py

Tasks designed to launch the PGD executable.

class vortex_cen.tasks.surfex.pgd.PgdCommonsMixin

Bases: SurfexCommonsMixin

Mixin methods for PGD binary IOs.

Configuration variables used by mixin methods:

• surfex_uenv or uenv The uenv that holds the soil databases for 2D simulations and the PGD executable.

• pgd_gvar optional variable to specify the name of the PGD executable in the uenv. Default is master_pgd_mpi if mpi=*True* and master_pgd_nompi` if mpi=*False*

• exesurfex path to the folder with surfex executables.

get_2D_databases()

Get Sand_DB.bin and Sand_DB.hdr, Clay_DB.bin and Clay_DB.hdr, ECOCLIMAP_II_EUROP.dir and ECOCLIMAP_II_EUROP.hdr from Uenv

Configuration variables used:

• surfex_uenv or uenv The uenv that holds the soil databases for 2D simulations.

get_mask_files()

Get water_mask and land_mask from Uenv

Configuration variables used:

• surfex_uenv or uenv The uenv that holds the soil databases for 2D simulations.

get_pgd_exe_from_local_path()

Fetch the PGD executable from a local path.

Configuration variables used:

• exesurfex path to the folder with surfex executables.

get_pgd_exe_from_uenv(mpi=True, fatal=True)

Method to get a PGD executable from uenv.

Parameters:

• mpi – True if an executable with MPI support should be fetched, False otherwise. Default is True.

• fatal – True if failing to fetch the executable should cause a fatal error, False otherwise. Default is True.

Configuration variables used:

• surfex_uenv or uenv The uenv that holds the PGD executable.

• pgd_gvar optional variable to specify the name of the PGD executable in the uenv. Default is master_pgd_mpi if mpi=*True* and master_pgd_nompi` if mpi=*False*

class vortex_cen.tasks.surfex.pgd._PgdConstruct(*args, **kw)

Bases: PgdCommonsMixin, _CenResearchTask

Task: _PgdConstruct

Abstract task for the generation of ground physiography (PGD.nc file).

Inputs:

• FORCING file

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• PGD executable

Outputs:

• PGD.nc (Ground physiography)

Mandatory configuration variables:

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• surfex_unev or uenv User Environment in which the following resources are to be retrieved:

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • PGD executable (unless supplied locally using the exesurfex variable).

  Format : uenv:{uenv_name}@{user} type: str

Optionnal configuration variables (other than forcing-specific ones):

• pgd_2d True if the PGD.nc should be calculated for a 2D domain. Default: False type: bool

• mask_2d True if you want to use land and water mask for a 2D domain. Default: False type: bool

• nprocs Number of process to allocate to the execution of the MPI binary. Default: 1 type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary. Default: 1 type: int

• nodes Number of nodes to allocate to the execution of the MPI binary. Default: 1 type: int

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

Forcing related configuration variables:

Mandatory:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

Optional:

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values : “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values : ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values : “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.pgd.MakePgd(*args, **kw)

Bases: _PgdConstruct

Task: MakePgd

Generation of ground physiography (PGD.nc file). Get PGD executable from Uenv

Inputs:

• FORCING file

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• PGD executable

Outputs:

• PGD.nc (Ground physiography)

Mandatory configuration variables:

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• surfex_unev or uenv User Environment in which the following resources are to be retrieved :

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • PGD executable (unless supplied locally using the exesurfex variable).

  Format : uenv:{uenv_name}@{user} type: str

Optionnal configuration variables (other than forcing-specific ones):

• pgd_2d True if the PGD.nc should be calculated for a 2D domain. Default: False type: bool

• mask_2d True if you want to use land and water mask for a 2D domain. Default: False type: bool

• exesurfex path to the folder with surfex executables (if supplied locally and not via the uenv).

• nprocs Number of process to allocate to the execution of the MPI binary. Default: 1 type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary. Default: 1 type: int

• nodes Number of nodes to allocate to the execution of the MPI binary. Default: 1 type: int

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

Forcing related configuration variables:

Mandatory:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.pgd.FetchPgdFileOrMake(*args, **kw)

Bases: _PgdConstruct

Task: FetchPgdFileOrMake

Generation of ground physiography (PGD.nc file). If PGD.nc is available in cache or archive for the current experiment fetch it. If not, try to get it from an uenv. If not either, generate it by calling the methods from the mother class.

WARNING : The simulation’s reproducibility can not be guaranteed with this task ! The PGD.nc file is only put to the vortex cache and not archived!

Inputs:

• FORCING file

• OPTIONS.nam ready-to-use SURFEX namelist (coming from an execution of a “Preprocess_Task”)

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• PGD executable

Outputs:

• PGD.nc (Ground physiography)

Mandatory configuration variables:

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• surfex_unev or uenv User Environment in which the following resources are to be retrieved :

  • ecoclimapI_covers_param.bin

  • ecoclimapII_eu_covers_param.bin

  • PGD executable (unless supplied locally using the exesurfex variable).

  Format : uenv:{uenv_name}@{user} type: str

Optional configuration variables (other than forcing-specific ones):

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID. defaults to xpid. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp. defaults to vapp. type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf. defaults to vconf. type: str

• pgd_vortex1 True if the pgd file was produced with vortex1 and uses vortex1 naming conventions. default: False type: bool

• pgd_user Name of the user who produced the PGD file. Default: None. type: str

• pgd_geometry geometry of the pgd file. Default: geometry. type: str, footprints.stdtypes.FPList

• pgdnc_gvar variable name of the pgd file in the uenv. Default: pgd_[geometry::tag].

• pgd_2d True if the PGD.nc should be calculated for a 2D domain. Default: False type: bool

• mask_2d True if you want to use land and water mask for a 2D domain. Default: False type: bool

• exesurfex path to the folder with surfex executables (if supplied locally and not via the uenv).

• nprocs Number of process to allocate to the execution of the MPI binary. Default: 1 type: int

• ntasks Number of tasks to allocate to the execution of the MPI binary. Default: 1 type: int

• nodes Number of nodes to allocate to the execution of the MPI binary. Default: 1 type: int

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

• diff_xpid Experiment id of the reference file used for reproducibility test. type diff_xpid: str

• diff_user user name associated with the reference file used for reproducibility test (only if different from current user). Default: None type diff_user: str

Forcing related configuration variables:

Mandatory:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

class vortex_cen.tasks.surfex.pgd.FetchPgdFileOrCrash(*args, **kw)

Bases: FetchPgdFileOrMake

Get a PGD.nc file from an uenv or vortex cache/archive. And put it in the cache of the current experiment. Crash if the file does not exist. The force_uenv configuration variable allows to look for the PGD.nc file exclusively in the uenv.

Outputs:

• PGD.nc (Ground physiography)

Mandatory configuration variables:

• surfex_uenv or if not present uenv User Environment from which the PGD.nc file should be fetched. Format : uenv:{uenv_name}@{user}

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier

Mandatory configuration variables unless ``force_uenv`` is *True*:

• pgd_xpid Experiment Identifier of the PGD file, if different from the task’s XPID. defaults to xpid. type: str

• pgd_vapp vapp of the PGD file, if different from the task’s vapp. defaults to vapp. type: str

• pgd_vconf vconf of the PGD file, if different from the task’s vconf. defaults to vconf. type: str

• pgd_vortex1 True if the pgd file was produced with vortex1 and uses vortex1 naming conventions. default: False type: bool

• pgd_user Name of the user who produced the PGD file. Default: None. type: str

Optional configuration variables:

• force_uenv If True the PGD.nc file must come from an uenv. Default: False

• pgdnc_gvar variable name of the pgd file in the uenv. Default: pgd_[geometry::tag].

• pgd_geometry geometry of the pgd file. Default: geometry. type: str, footprints.stdtypes.FPList

init_clim_ground_temperature.py

Tasks designed to generate an init_TG.nc file.

class vortex_cen.tasks.surfex.init_clim_ground_temperature.InitClimGroundTemperature(*args, **kw)

Bases: SurfexCommonsMixin, _CenResearchTask

Task: InitClimGroundTemperature

Initialize Surfex ground temperature (GT) by taking the climatological mean of the input forcing air temperature.

Inputs:

• FORCING file(s) on simulation geometry

Outputs:

• Init_TG file (initial values of ground temperature)

Mandatory configuration variables:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• geometry geometry of the current experiment. type geometry: str, footprints.stdtypes.FPList

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList (should be the same as geometry for this task, except if you have a very good reason why not.)

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

• xpid experiment id of the current experiment. Used to store the output.

• geometry geometry of the current experiment.

Optionnal configuration variables:

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

• out_block: block part of the vortex output path. Default: “prep” type out_block: str

• namespace_out Force specific namespace for output files (default: ‘vortex.multi.fr’) type: str

Configuration variables for reproducibility test

In general, the following variables will be used only for test cases.

• diff_xpid Experiment id of the reference file. type diff_xpid: str

• diff_user user name associated with the reference file (only if different from current user). Default: None type diff_user: str

• diff_block block part of the vortex path of the reference file. Default: “init_tg” type diff_block: str

class vortex_cen.tasks.surfex.init_clim_ground_temperature.FetchClimGroundTemperatureOrMake(*args, **kw)

Bases: InitClimGroundTemperature

Task: FetchClimGroundTemperatureOrMake

If InitTG is available in cache or archive for the current experiment fetch it. If not, try to get it from an uenv. If not either, generate it by calling the methods from the mother class.

Configuration variables used for fetching from cache or archive

• tg_xpid or xpid experiment id the init_TG.nc file should be fetched from.

• tg_user name of the user that produced the target the init_TG.nc file. Default: None

• tg_geometry or geometry geometry of the init_TG. Logically the same as for the rest of the simulation

• tg_vapp or vapp Application name to search the init_TG.nc file.

• tg_vconf or vconf Configuration name to search the init_TG.nc file.

Configuration variables used for fetching from uenv:

• tg_geometry or geometry geometry of the init_TG. Logically the same as for the rest of the simulation

• surfex_uenv or if not present uenv User Environment from which the init_TG.nc file should be fetched. Format : uenv:{uenv_name}@{user}

• tg_gvar key to look up the init_TG.nc file in the uenv the file should come from.

Configuration variables used for calculating initial ground temperature:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• geometry geometry of the current experiment. type geometry: str, footprints.stdtypes.FPList

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList (should be the same as geometry for this task, except if you have a very good reason why not.)

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

• xpid experiment id of the current experiment. Used to store the output.

• geometry geometry of the current experiment.

Optional

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

• out_block: block part of the vortex output path. Default: “prep” type out_block: str

Configuration variables for reproducibility test

In general, the following variables will be used only for test cases.

• diff_xpid Experiment id of the reference file. type diff_xpid: str

• diff_user user name associated with the reference file (only if different from current user). Default: None type diff_user: str

• diff_block block part of the vortex path of the reference file. Default: “init_tg” type diff_block: str

class vortex_cen.tasks.surfex.init_clim_ground_temperature.MakeClimGroundTemperatureIfNoPrep(*args, **kw)

Bases: FetchClimGroundTemperatureOrMake

Task: MakeClimGroundTemperatureIfNoPrep

If the “climground” is provided and set to “True”, this task will look for a PREP.nc file and if none is found, it will initialize Surfex ground temperature (GT) by taking the climatological mean of the input forcing air temperature.

Inputs:

• FORCING file(s) on simulation geometry

Outputs:

• Init_TG file (initial values of ground temperature)

Optional configuration variables

• climground Allow the generation of a ground initialization file by computing a climatological average of air temperature on the provided period. Default: False type climground: bool

Configuration variables if ``climground`` is ``True``:

Search for PREP file

• prep_xpid or xpid Experiment id the prep file should be searched for or put in cache.

• prep_user name of the user who produced the PREP file. Default: None.

• prep_datevalidity Validity date of the prep file.

• prep_vapp or vapp Application name to search the PREP.nc file.

• prep_vconf or vconf Configuration name to search the PREP.nc file.

• prep_vortex1 type: bool. True if the requested PREP.nc file was produced with vortex 1 and thus uses vortex 1 naming conventions. Default is False.

• prep_geometry or geometry geometry of the PREP file.

• prep_namebuild Default: flat@cen

• prep_block block part of the data tree to search for the PREP.nc file. Default is prep.

• prep_member or member If the PREP.nc file comes from an ensemble, a member can be chosen. Default is None.

• prep_cutoff Can be used to select a PREP file coming from an operational forecast (forecast) or analysis (assimilation). Default is None. Might be useful for reforecasts.

Search for initTG file in archive:

• tg_xpid or xpid experiment id the init_TG.nc file should be fetched from.

• tg_user name of the user that produced the target the init_TG.nc file. Default: None

• tg_geometry or geometry geometry of the init_TG. Logically the same as for the rest of the simulation

• tg_vapp or vapp Application name to search the init_TG.nc file.

• tg_vconf or vconf Configuration name to search the init_TG.nc file.

Search for initTG file in unev:

• tg_geometry or geometry geometry of the init_TG. Logically the same as for the rest of the simulation

• surfex_uenv or if not present uenv User Environment from which the init_TG.nc file should be fetched. Format : uenv:{uenv_name}@{user}

• tg_gvar key to look up the init_TG.nc file in the uenv the file should come from.

Calculate initial ground temperature:

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• geometry geometry of the current experiment. type geometry: str, footprints.stdtypes.FPList

• forcing_geometry geometry footprint, default self.conf.geometry type forcing_geometry: str, footprints.stdtypes.FPList (should be the same as geometry for this task, except if you have a very good reason why not.)

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• forcing_namespace namespace footprint, default “vortex.multi.fr” (hendrix + local cache) type forcing_namespace: str

• forcing_date date footprint (unsed with the research namebuilders), default to [dateend] type forcing_date: str

• forcing_model model footprint (to be made optional for SurfaceIO objects), default None type forcing_model: str

• xpid experiment id of the current experiment. Used to store the output.

• geometry geometry of the current experiment.

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

• out_block: block part of the vortex output path. Default: “prep” type out_block: str

class vortex_cen.tasks.surfex.init_clim_ground_temperature.FetchClimGroundTemperatureOrCrash(*args, **kw)

Bases: InitClimGroundTemperature, _CenResearchTask

Try to get the ground temperature climatology file from an uenv or potentially from the archive and put it to the cache. Crash if the file is not available.

Mandatory configuration variables:

• surfex_uenv or if not present uenv User Environment from which the PGD.nc file should be fetched. Format : uenv:{uenv_name}@{user}

• tg_gvar key to look up the init_TG.nc file in the uenv the file should come from.

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier

Mandatory configuration variables unless ``force_uenv`` is *True*:

• tg_xpid or xpid experiment id the init_TG.nc file should be fetched from.

• tg_user name of the user that produced the target the init_TG.nc file. Default: None

• tg_geometry or geometry geometry of the init_TG. Logically the same as for the rest of the simulation

• tg_vapp or vapp Application name to search the init_TG.nc file.

• tg_vconf or vconf Configuration name to search the init_TG.nc file.

Optional configuration variables:

• force_uenv If True the Init_TG.nc file must come from an uenv. Default: False

soda.py

Tasks designed to launch the SODA executable.

class vortex_cen.tasks.surfex.soda.SodaCommonsMixin

Bases: SurfexCommonsMixin

Mixin methods for SODA binary IOs.

class vortex_cen.tasks.surfex.soda.Soda(*args, **kw)

Bases: SodaCommonsMixin, _CenResearchTask

Task: Soda

SODA Particle Filter assimilation task. Reference : Cluzet et al. (2021): https://gmd.copernicus.org/articles/14/1595/2021/

Input:

• SODA namelist (OPTIONS.nam)

• Ensemble of snowpack initial conditions (“PREP.nc”) refered to as “background”

• ecoclimapI_covers_param.bin and ecoclimapII_eu_covers_param.bin (binaries for vegetation generation)

• drdt_bst_fit_60.nc (Crocus metamorphism parameters)

• PGD.nc (Ground physiography)

Outputs:

• Modified ensemble of snowpack initial conditions (“PREP.nc”) referred to as “analysis”

Mandatory configuration variables

• date or assimdate Analysis date type: str, Date

• geometry geometry of the PREP files type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• uenv User Environment in which the following resources are to be retrieved: Format : uenv:{uenv_name}@{user} type: str

• observation_block By convention, the sensor used for the observation (ex: MODIS, PLEIADES, VIIRS) type: str

• members Ensemble members type: footprints.stdtypes.FPList

Optional configuration variables

• scope Scope of the observation type: str

• observation_xpid Experiment identifier of the observation file type: str

• observation_user User who produced / owns the observation file type: str

• observation_vapp vapp of the observation file type: str

• observation_vconf vconf of the observation file type: str

• prep_vapp vapp of the PREP files type: str

• prep_vconf vconf of the PREP files type: str

• prep_xpid Exepriment Identifier of the PREP files type: str

• prep_user User who produced the PREP files type: str

• prep_vortex1 Whether or not the PREP files have been produced with Vortex-v1 type: bool

• pgd_vapp vapp of the PGD file type: str

• pgd_vconf vconf of the PGD file type: str

• pgd_xpid Exepriment Identifier of the PGD file type: str

• pgd_user User who produced the PGD file type: str

• pgd_vortex1 Whether or not the PGD files have been produced with Vortex-v1 type: bool

class vortex_cen.tasks.surfex.soda.FetchBackgroundOrCrash(*args, **kw)

Bases: SodaCommonsMixin, _CenResearchTask

Fetch an ensemble of PREP files from the archive as background for a SODA execution and put it in the cache of the current experiment.

Mandatory configuration variables:

• geometry geometry of the forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• date or assimdate Analysis date type: str or Date

Optional configuraiton variables

• prep_xpid or xpid Experiment id the prep file should be searched for or put in cache.

• prep_user name of the user who produced the PREP file. Default: None.

• prep_vapp or vapp Application name to search the PREP.nc file.

• prep_vconf or vconf Configuration name to search the PREP.nc file.

• prep_vortex1 type: bool. True if the requested PREP.nc file was produced with vortex 1 and thus uses vortex 1 naming conventions. Default is False.

safran

Generation of SAFRAN forcing files.

SAFRAN input mixin

class vortex_cen.tasks.safran.common.SafranMixIn

Bases: object

get_const_safran()

Get Safran constant files.

Configuration variables used:

• cycle genv with safran constant files.

• geometry dict with possible geometries. type: dict

• vconf actual geometry

reanalysis.py

class vortex_cen.tasks.safran.reanalysis.SafranReanalysis(*args, **kw)

Bases: _CenResearchTask

Task : SafranReanalysis

SAFRAN reanalysis.

Reference : https://essd.copernicus.org/articles/14/1707/2022/ Associated opensource dataset : https://doi.org/10.25326/37#v2020.2

Input:

• Guess : daily packed files containing all lead time of a given 0H run of ARPEGE / PERAP

• Observations : Packed SAFRAN-readable surface observation files (R, S and T files)

• listem : List of SAFRAN massifs

• listeml : List of coordinates of the SAFRAN massifs

• Listeo : list of potential observation sites to assimilate

• NORELmt : Monthly mean precipitation value

• rsclim / icrccm : Climatological values

• ADAPT/ANALYSE/EBAUCHE/IMPRESS/MELANGE/SORTIES : Safran namelists

• carpost.tar : Files describing the output “postes”

• safrane : Safran executable for synoptic interpolation of the guess on the Safran geometry

• syrpluie / syrmRR : Safran executables for precipitation spatio-temporal precipitation interpolation

• sytist : Safran executable for hourly interpolation and the creation of FORCING files

Output:

• FORCING_massifs.nc : Ensemble of forcing files on the “flat” massif geometry

• FORCING_postes.nc : Ensemble of forcing files on the “postes” geometry

• listings_safran : output safran execution listings

• liste_obs : List of assimilated observations

Mandatory Configuration Variables:

• datebegin First rundate of the guess (hour must be ‘00’)

• dateend Last run date of the guess (hour must be ‘00’)

• xpid Experiment id. Do not use experiment ids with 4 letters.

• geometry Geometry of the simulation. This must be a valid geometry in your ‘$HOME/.vortexrc/geometries.ini’ file.

• guess_geometry Geometry of SAFRAN guess files. type: dict

• obs_geometry geometry of the observation files. type: dict

• uenv Name of the UEnv containing all SAFRAN constant input files and executables

• ntasks Number of parallel tasks to allocate to the execution. type: int or dict[geometry]

• nnodes Number of nodes to allocate to the execution. type: int or dict[geometry]

• execution Type of SAFRAN execution. type: str, choices: analysis, forecast, reanalysis, reforecast

• assim Allow assimilation of observations. type: bool

Optional Configuration Variables:

• obs_xpid Experiment identifier of the observation files. type: str, default: xpid

• obs_vapp vapp level of the observation files. type: str, default: vapp

• obs_vconf vconf level of the observation files. type: str, default: vconf

• obs_user Name of the producer of the observation files. type: str

• diff_xpid Experiment identifier of the reference files for reproducibility check. type: str

• diff_user Name of the producer of the reference files for reproducibility check. type: str

reconstruct_obs_safran.py

class vortex_cen.tasks.safran.reconstruct_obs_safran.ReconstructSafranObs(*args, **kw)

Bases: _CenResearchTask

Task : Reconstruct_SAFRAN_Obs

Task to build SAFRAN-compatible hourly observations files from reconstructed observation series.

Input:

• NEW_OBSERVATIONS.nc : file containing reconstructed hourly temperature “observations”

• OBSERVATIONS.tar : archive containing real Safran-compatible surface observation files (R, S and T files)

• listeo file : providing the metadata of all observation sites

Output:

• OBSERVATIONS.tar : archive containing Safran-compatible files with both real and reconstructed observations

Mandatory Configuration Variables:

• datebegin First rundate of the guess (hour must be ‘00’)

• dateend Last run date of the guess (hour must be ‘00’)

• xpid Experiment id. Do not use experiment ids with 4 letters.

• geometry Geometry of the simulation. This must be a valid geometry in your ‘$HOME/.vortexrc/geometries.ini’ file.

• uenv Name of the UEnv containing all SAFRAN constant input files and executables.

Optional Configuration Variables:

• newobs_xpid Experiment identifier of the reconstructed hourly temperature observation dataset. type: str, default: xpid

• newobs_user Username of the producer of the reconstructed hourly temperature observation dataset. type: str, default: $USER

prep_reforecast.py

class vortex_cen.tasks.safran.prep_reforecast.PrepSafran(*args, **kw)

Bases: _CenResearchTask

Task : PrepSafran

Generation of guess files for the Safran ensemble re-forecast task (daily run covering J 6H –> J+4 6H). SAFRAN guess files come from both the PEARP ensemble and ARPEGE (as member ‘N+1’) from the 0 UTC run.

Input:

• METADATA: Description of the ARPEGE / PEARP grib files grid / geometry

• ARPEGE.grib: ARPEGE forecasts

• PEARP.grib: PEARP ensemble forecasts

• massifs_safran.tar: shapefile describing the Safran massifs

• makeP.py: script generating the Safran guess files from the ARPEGE / PEARP forecasts

Output:

• PYYMMDDHH: Safran guess files, grouped in tar archives containing all ‘P’ files from all members and lead times of a given model run

Mandatory configuration variables:

• datebegin First rundate of the guess (hour must be ‘00’)

• dateend Last run date of the guess (hour must be ‘00’)

• xpid Experiment id. Do not use experiment ids with 4 letters.

• geometries List of output geometries of the simulation (these must be valid geometry tags in your ‘$HOME/.vortexrc/geometries.ini’ file.

• uenv Name of the UEnv containing constant files (METADATA, massifs_safran.shp and makeP.py script)

• arpege_geometry Geometry of ARPEGE analyses / forecasts files used to generate Safran guess type: str

• pearp_geometry Geometry of PEARP forecast files used to generate Safran guess type: str

• prv_terms Lead times of the Safran guess files. type: footprints.stdtypes.FPList, format = first-last-step

• nwp_xpid Experiment identifier of the NWP models used to generate Safran guess files

• members The list of ensemble members. Default: None

• ntasks Number of parallel tasks to allocate to the execution. type: int or dict[geometry]

• nnodes Number of nodes to allocate to the execution. type: int or dict[geometry]

reforecast.py

class vortex_cen.tasks.safran.reforecast.SafranReforecast(*args, **kw)

Bases: _CenResearchTask

Task : SafranReforecast

Safran ensemble re-forecast task (daily run covering J 6H –> J+4 6H). SAFRAN guess files come from both the PEARP ensemble and ARPEGE (as member ‘N+1’) from the 0 UTC run.

Input:

• Guess: daily packed files containing all lead time of a given 0H run of ARPEGE / PERAP

• listem: List of SAFRAN massifs

• listeml: List of coordinates of the SAFRAN massifs

• Listeo: list of potential observation sites to assimilate (mandatory but unused)

• NORELmt: Monthly mean precipitation value

• rsclim / icrccm: Climatological values

• ADAPT/ANALYSE/EBAUCHE/IMPRESS/MELANGE/SORTIES: Safran namelists

• carpost.tar: Files describing the output “postes”

• safrane: Safran executable for synoptic interpolation of the guess on the Safran geometry

• syrpluie / syrmRR: Safran executables for precipitation spatio-temporal precipitation interpolation

• sytist: Safran executable for hourly interpolation and the creation of FORCING files

Output:

• FORCING_massifs.nc: Ensemble of forcing files on the “flat” massif geometry

• FORCING_postes.nc: Ensemble of forcing files on the “postes” geometry

Mandatory Configuration Variables:

• datebegin First rundate of the guess (hour must be ‘00’)

• dateend Last run date of the guess (hour must be ‘00’)

• xpid Experiment id. Do not use experiment ids with 4 letters.

• geometry Geometry of the simulation. This must be a valid geometry in your ‘$HOME/.vortexrc/geometries.ini’ file.

• uenv Name of the UEnv containing all SAFRAN constant input files and executables

• prv_terms Lead times of the Safran guess files. type: footprints.stdtypes.FPList, format = first-last-step

• ntasks Number of parallel tasks to allocate to the execution. type: int or dict[geometry]

• nnodes Number of nodes to allocate to the execution. type: int or dict[geometry]

• `` members`` The list of ensemble members. Default: None

• execution Type of SAFRAN execution. type: str, choices: analysis, forecast, reanalysis, reforecast

• assim Allow assimilation of observations. type: bool

Optional Configuration Variables:

• guess_xpid Experiment identifier of the SAFRAN guess files. type: str Default: xpid

• guess_user Username of the producer of the SAFRAN guess files. type: str, default: $USER

regrid

Tasks modifying a forcing file into a SURFEX-ready FORCING file.

add_slopes.py

Add slopes to forcing file(s) in the SAFRAN “flat” massif geometry.

class vortex_cen.tasks.regrid.add_slopes.AddSlopes(*args, **kw)

Bases: _CenResearchTask

Task : AddSlopes Add slopes to forcing file in a “flat” geometry.

Input:

• SAFRAN-generated FORCING file in a “flat” geometry.

Output:

• FORCING file wih slopes and aspects.

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s) type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s) type: str, footprints.stdtypes.FPList

• forcing_geometry geometry of the input forcing file(s) type: str, footprints.stdtypes.FPList

• geometry geometry of the output forcing file(s) type: str, footprints.stdtypes.FPList

• xpid Experiment identifier type: str

Optional configuration variables:

• max_ntasks The maximum number of parallel tasks (in case of huge memory usage) type: int

concatenate.py

class vortex_cen.tasks.regrid.concatenate.ForcingSpatialConcatenation(*args, **kw)

Bases: _CenResearchTask

Concatenate FORCING files over a spatial dimension (Typically “Number_of_points”). FORCING files over different years are processed in parallel. Ex : concatenation of “postes” simulations over different mountain ranges

Input:

• Set of FORCING files to concatenate

Output:

• Single FORCING file

Mandatory Configuration variables:

• forcing_geometry List of geometries of the FORCING files to concatenate type: footprints.stdtypes.FPList

• xpid Experiment identifier type: str

• geometry Geometry of the output file(s) type: str

• datebegin begin date(s) of files

• dateend end date(s) of files

• namespace_out namespace of output files

Optional configuration variables:

• max_ntasks The maximum number of parallel tasks (in case of huge memory usage) type: int

• block Output block of the concatenated FORCING files type: str

extract_s2m_points.py

class vortex_cen.tasks.regrid.extract_s2m_points.ExtractS2MForcing(*args, **kw)

Bases: _CenResearchTask

Parallel extraction of a list of points from an ensemble of FORCING file(s) covering different time periods in the “massif” geometry according to their massif number, elevation, slope and aspect.

Input:

• SAFRAN-generated FORCING file(s) in the “massif” geometry.

Outputs:

• FORCING file(s) with extracted points

Configuration variables:

• massifs Massif number(s) to be extracted type: int, list

• slopes Slope(s) to be extracted type: int, list

• elevations Elevations(s) to be extracted type: int, list

• aspects Aspects(s) to be extracted type: int, list

• geometry Geometry of the output file(s) type: str

• xpid Experiment identifier type: str

extract_subperiod.py

class vortex_cen.tasks.regrid.extract_subperiod.ExtractSubPeriod(*args, **kw)

Bases: _CenResearchTask

Extract a sub period in a Forcing file

Inputs:

• FORCING file

Outputs:

• FORCING file on a shorter period

Configuration variables:

• datebegin begin date(s) of files

• dateend end date(s) of files

• datebegin begin date(s) of files

• dateend end date(s) of files

• forcing_geometry geometry of the forcing file which is going to be time cut

interpol.py

class vortex_cen.tasks.regrid.interpol.InterpolateS2MForcing(*args, **kw)

Bases: _CenResearchTask, InterpolMixIn

Interpolate a forcing file in “massif” geometry onto a 2D grid, or 1D grid, that is a list of points.

Inputs:

• FORCING file in the “massif” geometry.

• GRID file containing the desired output grid.

• interpolation binary

Outputs:

• FORCING file on the new grid.

Configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry of input file type: str, footprints.stdtypes.FPList

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• gridout path to output grid file type: str, pathlike

• uenv environment containing the interpolation executable

• xpid Experiment identifier type: str

• geometry Geometry of the output file(s) type: str

• datebegin begin date(s) of files

• dateend end date(s) of files

• namespace_out namespace of output files

Optional

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values : “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values : ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values : “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

• out_block block part of the output directory. Default: interpol

• diff_xpid Experiment identifier of the reference experiment in case of reproducibility check

• diff_user vortex user of the reference experiment

• diff_block block part of the reference file directory. Default: interpol

class vortex_cen.tasks.regrid.interpol.InterpolMixIn

Bases: object

Methods for interpolation task

get_interpolation_binary()

Get the interpolation binary from the uenv

Configuration variables used:

• uenv

get_output_grid_definition()

get output grid file.

Configuration variables used:

• uenv

• gridout The variable name of the output grid file in the uenv (gvar). Default is DEM.

class vortex_cen.tasks.regrid.interpol.InterpolateS2MLocalForcing(*args, **kw)

Bases: InterpolateS2MForcing

Interpolate a forcing file in “massif” geometry onto a 2D grid, or 1D grid, that is a list of points.

Inputs:

• Local FORCING file in the “massif” geometry.

• GRID file containing the desired output grid.

• interpolation binary

Outputs:

• FORCING file on the new grid.

Configuration variables:

Mandatory

• forcing_datebegin datebegin footprint, default self.conf.datebegin type forcing_datebegin: str, footprints.stdtypes.FPList

• forcing_dateend dateend footprint, default self.conf.dateend type forcing_dateend: str, footprints.stdtypes.FPList

• forcing_xpid Experiment identifier, default self.conf.xpid type forcing_xpid: str

• forcing_geometry geometry of input file type: str, footprints.stdtypes.FPList

• forcing_block block footprint, default “meteo” type forcing_vconf: str

• gridout path to output grid file type: str, pathlike

• uenv environment containing the interpolation executable

• xpid Experiment identifier type: str

• geometry Geometry of the output file(s) type: str

• datebegin begin date(s) of files

• dateend end date(s) of files

• namespace_out namespace of output files

Optional

• forcing_vapp vapp footprint, default self.conf.vapp type forcing_vapp: str

• forcing_vconf vconf footprint, default self.conf.vconf type forcing_vconf: str

• forcing_member member footprint, default None (or member if provided) type forcing_member: int, footprints.stdtypes.FPList

• forcing_namebuild namebuild footprint, default “flat@cen” (will change soon) type forcing_namebuild: str

• forcing_intent intent footprint (local file permissions), default “in” Possible values: “in” (read-only), “inout” (read-write) type forcing_intent: str

• forcing_source_app source_app footprint, default None type forcing_source_app: str, footprints.stdtypes.FPList

• forcing_source_conf source_conf footprint, default None type forcing_source_conf: str, footprints.stdtypes.FPList

• forcing_source Retrieve source_app and source_conf footrprints dictionnaries for S2M reanalysis Possible values: ‘era5’, ‘era40’ type forcing_source: str

• forcing_cutoff cutoff footprint (to be made optional for SurfaceIO objects), default None type forcing_cutoff: str

• io_duration Argument similar to the one of the get_list_dates_files method in snowtools/utils/dates.py. Used to retrieve the list of datebegin and dateend for inputs covering sub-periods. Possible values: “yearly”, “monthly” or “full” type io_duration: str

• forcing_vortex1 Boolean to identify resources produced with vortex1 (filename without geometry) type forcing_vortex1: bool

• out_block block part of the output directory. Default: interpol

• diff_xpid Experiment identifier of the reference experiment in case of reproducibility check

• diff_user vortex user of the reference experiment

• diff_block block part of the reference file directory. Default: interpol

shadows.py

class vortex_cen.tasks.regrid.shadows.Shadows(*args, **kw)

Bases: _CenResearchTask

Task : Shadows

Add relief-induced solar masks to a FORCING file in a “station” geometry.

Input:

• SAFRAN-generated FORCING file in the “station” geometry.

Output:

• FORCING file with extracted solar masks added.

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• forcing_geometry geometry of the input forcing file(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the output forcing file(s). type: str, footprints.stdtypes.FPList

• xpid Experiment identifier. type: str

class vortex_cen.tasks.regrid.shadows.ShadowsPostes(*args, **kw)

Bases: Shadows

In the reanalysis case, the FORCING files come from the output of the “concatenation” task and are not available at the execution of the transfer node.

Mandatory configuration variables:

• datebegin datebegin of the forcing file(s). type: str, footprints.stdtypes.FPList

• dateend dateend of the forcing files(s). type: str, footprints.stdtypes.FPList

• forcing_geometry geometry of the input forcing file(s). type: str, footprints.stdtypes.FPList

• geometry geometry of the output forcing file(s). type: str, footprints.stdtypes.FPList

• xpid Experiment identifier. type: str