Welcome to ClimdexCalc’s documentation!

The ClimDexCalc Plugin

authors:

Andy Richling [1], Igor Kröner, Jens Grieger, Christopher Kadow

Institut für Meteorologie, Freie Universität Berlin

Version from 2023-09-27

Link to gitlab repository

This Plugin uses the ClimDex software provided by UNSW Australia and Pacific Climate Impacts Consortium (PCIC) to calculate extreme indices. Based on the R-package climdex.pcic [2] and climdex.pcic.ncdf [3] several indices for temperature and precipitation are estimated for a specified dataset (reanalysis, historical, decadals) and optionally linked to Freva for additional calculation.

Note

The Plugin was originally developed by Igor Kröner for the application in the context of decadal prediction experiments. Adjustments that go beyond the scope of the decadal predictions are in progress and still under development.

Warning

Depending on the gridtype of the data and chosen percentile_method there might be bilinear remapping into regular lonlat grid using cdo, previous to the indices calculation:

gridtype	percentile_method	mapping
gaussian	climdex-pcic	no
curvilinear	climdex-pcic	yes (scripts/CalcIndices.R)
gaussian	decadal-eval	yes (scripts/CalcThresholds.sh)
curvilinear	decadal-eval	not handled

This means that indices produced with climdexcalc may slightly differ from equally named indices hosted at, for example, CDS or produced by Climpact-SCI plugin.

Introduction

In the context of changes in climate extremes it might be useful to have some standardised parameters to compare temperature and precipitation extremes between different datasets. For this, the joint CCl/CLIVAR/JCOMM Expert Team (ET) on Climate Change Detection and Indices (ETCCDI) provide a software to calculate 27 several climate extreme indices on the basis of daily temperature and precipitation data (http://etccdi.pacificclimate.org/list_27_indices.shtml) [4]. The ClimDex software given by the R-packages climdex.pcic and climdex.pcic.ncdf is implemented as a Plugin for the Central Evaluation System (CES) and can be used for observations, reanalyses, historical and future simulations as well as for decadal experiments. Depending on the extreme index, monthly and/or annual values with respect to frequency, duration or amplitude aspects are calculated.

User Manual

At first, the input data from data-browser via the options project, product, institute, model, experiment, and ensemble must be selected. In case the data is a decadal experiment, is_decadal is set to True.

Next, the timeperiod of interest which should be analysed have to be given in the comma-separated format firstyear,lastyear. In case of decadal experiments, this options defines the first and last year of initialisation. The parameter baseperiod defines the base period for percentile-based climate indices. In case of decadal experiments, the base period is referred to lead year 1. In time_frequency_output the temporal aggregation level of the resulting output can be chosen. Some indices are only available for annual aggregation level. Please check before or use the selection all.

In indices, the climate indices (http://etccdi.pacificclimate.org/list_27_indices.shtml) in a comma-separated format can be selected. The data-browser should be checked to ensure the index-related variables tas, tasmax, tasmin and/or pr are available for the chosen input data. Note, tx10p/tx90p or tn10p/tn90p or r90p/r95p can not be calculated individually and need to be kept in pairs. In case of decadal experiments, the calculation of duration based indices (wsdi,csdi,cdd,cwd) are not supposed be combined with percentile indices. For percentile-based indices the method of threshold-calculation can be selected in the option percentile_method. More information about the methods can be found in section 4. In case already calculated thresholds are to be used, an existing threshold file calculated via the Plugin can be specified in percentilefile. In case of multiple files for tasmin/tasmax/pr-percentiles the name of the variable must be substituted with xxxxx. If lead time dependent percentiles are given, lead time is substituted with yyyyy. (e.g. /link/to/your/file/threshold_xxxxx_day_mpi-esm-lr_historical_r1i1p1_1961-1990_ltyyyyy.nc)

Finally, the number of parallel-processing tasks (ntask), the output (outputdir) and cache (cachedir) directories have to be specified. Further the options to just show founded files for processing (dryrun), to remove the cache directories (cacheclear) and to link the resulting data into the system (link2database) can be used.

Input Parameter

freva plugin climdexcalc --doc
ClimDexCalc (v1.0.1): This Plugin uses the ClimDex software provided by UNSW
Australia and PCIC to calculate climate extreme indices suggested by the joint
CCl/WCRP/JCOMM Expert Team on Climate Change Detection and Indices (ETCCDI).
Based on the R-packages climdex.pcic and climdex.pcic.ncdf several indices for
temperature and precipitation are estimated for a specified dataset
(reanalysis, historical, decadals) and (optionally) linked to the
Central Evaluation System for further calculation.
Options:
project                 (default: <undefined>) [mandatory]
                        Choose project e.g. CMIP5, Reanalysis, your data, etc.
                        (check data-browser)
product                 (default: <undefined>) [mandatory]
                        Choose product (check data-browser)
institute               (default: <undefined>) [mandatory]
                        Choose institute (check data-browser)
model                   (default: <undefined>) [mandatory]
                        Choose model (check data-browser)
experiment              (default: <undefined>) [mandatory]
                        Choose experiment (check data-browser)
ensemble                (default: <undefined>) [mandatory]
                        Choose ensemble member - multi selection possible (comma
                        separated), use * for all members
cmor_table              (default: <undefined>) [mandatory]
                        Choose cmor table
is_decadal              (default: False)
                        Set "True" in case forecast experiment is a decadal
                        experiment.)
timeperiod              (default: 1979,2010) [mandatory]
                        Decadal Experiment: First and last year of
                        Initialization (comma separated) of the time period of
                        interest.
                        Others: First and last year (comma separated) of the
                        time period of interest.
baseperiod              (default: <undefined>)
                        First and last year (comma separated) of base period. If
                        percentile-based-indices are selected, this period is
                        used for their calculation. It is suggested to use
                        1961,1990 to guarantee comparability to other studies.
                         If "is_decadal" is "True" base period is referred to
                        lead time 1.
                         If nothing is selected but percentile-indices are
                        calculated the whole time period is used as basis for
                        percentile calculation.
time_frequency_output   (default: all) [mandatory]
                        Temporal resolution of the output time series you want
                        to analyze.
                        Valid options are "annual", "monthly" and "all" (both
                        monthly and annual).
                         Note: Some indices are only available for "annual".
                        Please check before!
indices                 (default: tx10p,tx90p,tn10p,tn90p,txx,txn,tnx,tnn,id,fd,
                         su,tr,dtr,prcptot,rx5day,rx1day,r10mm,r20mm,r95p,r99p,sdii)
                        Comma separated list of indices that will be calculated.
                        Default are all - except duration indices. If you only
                        need some, delete the others. See www.climdex.org for
                        description of the indices.
                         Check data-browser to ensure the index-related
                        variables "tas", "tasmax", "tasmin" and/or "pr" are
                        available for the chosen input data!
                         Be aware of
                         • tx10p/tx90p or tn10p/tn90p or r90p/r95p  can not be
                        calculated individually and need to be kept in pairs!
                         • FOR DECADALS: Calculation of duration based indices
                        (wsdi,csdi,cdd,cwd) are not supposed be combined with
                        percentile indices!
percentile_method       (default: climdex-pcic) [mandatory]
                        Method of calculating percentile-based threshold.
                        climdex-pcic: Original method from the R-Package
                        climdex.pcic
                         decadal-eval: Method adapted for decadal evaluation.
percentilefile          (default: <undefined>)
                        Optional link to an existing percentile file. If several
                        files for tasmin/tasmax/pr-percentiles then substitute
                        the name of the variable xxxxx. If lead time dependent
                        percentiles are given, substitute the lead time with
                        yyyyy. e.g. /link/to/your/file/threshold_xxxxx_day_mpi-
                        esm-lr_historical_r1i1p1_1961-1990_ltyyyyy.nc
ntask                   (default: 6) [mandatory]
                        Choose number of tasks on machine, default set by Freva
                        for batchmode.
dryrun                  (default: False)
                        Whether or not only data-search should be done.
outputdir               (default: $USER_OUTPUT_DIR/$SYSTEM_DATETIME)
                        Choose your output directory. Default is set by Freva.
cachedir                (default: $USER_CACHE_DIR)
                        Choose your cache directory. Default is set by Freva.
cacheclear              (default: True)
                        Option switch to NOT clear the cache.
link2database           (default: False)
                        Option whether the output data should be linked or not.
extra_scheduler_options (default: )
                        Set additional options for the job submission to the
                        workload manager (, seperated). Note: batchmode and web
                        only.

Output Parameter

The resulting files contain the selected climate indices – one individual file for each index. The files follow the CMOR structure of the system, where the variable of a specific index is named as INDEXetccdi (e.g. tx10petccdi_yr_reanalysis_eraint_r1i1p1_1980-2000.nc for climate index tx10p). In case percentile-based indices are calculated and no threshold file is given by the user, the calculated threshold file is located in the cache directory if the option cacheclear is set to False.

Previous Applications and Test

The Plugin was used to compare climate extreme indices of decadal experiments with respect to reanalysis data.

Scientific Background

The detailed description of the climate indices can be found on the website http://etccdi.pacificclimate.org/list_27_indices.shtml. For the indices using thresholds based on percentiles (tx10p, tx90p, tn10p, tn90p, wsdi, csdi, r95p, r99p) two method for calculating the percentiles (percentile_method) can be applied in case no percentile file is given by the user. The first method climdex-pcic uses the originally implemented R-routine from the software, while the second method (decadal-eval) is based on a routine especially adapted for decadal experiments. Nevertheless, both methods can be applied to non-decadal experiments, depending on the needs of the user. The substantive and technical differences between the two methods are listed below.

• climdex-pcic:

  • percentile calculation based on the routines implemented in the R-package climdex.pcic.ncdf

  • percentile of a calendar day is centred on a 5-day window with respect to the given base period using the R-quantile function type 8

  • calculation of percentiles separately for years inside and outside the base period [5], but the calculated percentile file just contains the out-base values

  • in case of an ensemble: percentiles are calculated individually for each member

• decadal-eval:

  • percentile of a calendar day is centred on a 5-day window with respect to the given base period using the R-quantile function type 7

  • calculation of percentiles is NOT separated for years within and outside the base period

  • in case of an ensemble: percentiles are calculated on the basis of all members

Additional Information, Remarks and Installation

Installation

Software:

• Python(>=3.10)

• NetCDF4

• udunits

• CDO (>=2.0.5)

• R (>=4.1.3)

• relevant R-packages plus dependencies: climdex.pcic, climdex.pcic.ncdf, ncdf4, snow, Rcpp, caTools, PCICt, SPEI, functional, udunits2

Installation Steps:

1. Get the Plugin from the git repository:

   git clone https://gitlab.dkrz.de/freva/plugins4freva/climdexcalc.git
   cd climdexcalc
   

2. Create the conda environment:

   1. you will need to have a suitable version of conda active previously, for example by loading one Freva instance, e.g.:

   module load clint <your-projects-freva>
   

   2. run the Makefile:

   make all
   

To activate/deactivate the environment including own CDO and R compiler

(not necesary to run the plugin via Freva):

source plugin_env/bin/activate # activate
conda deactivate # deactivate

Updating the Plugin with git:

1. cd <PATH2PLUGIN>

2. git checkout -b <new_branch> (commit changes in a new branch <new_branch>)

3. git pull -u <new_branch> (pull changes upstream to <new_branch>)

4. pull/merge request into main/master branch (at e.g. gitlab), wait for reviews, merge

Contact & References

[1]

andy.richling@met.fu-berlin.de

[2]

David Bronaugh 2020 - PCIC Implementation of Climdex Routines - R package version 1.1-11

[3]

David Bronaugh 2017 - Functions to compute CLIMDEX indices over a NetCDF grid - R package version 0.5-4

[4]

Karl et al. 1999 - CLIVAR/GCOS/WMO Workshop on Indices and Indicators for Climate Extremes - doi: 0.1023/A:1005491526870

[5]

Zhang et al. 2005 - Avoiding Inhomogeneity in Percentile-Based Indices of Temperature Extremes - doi: 10.1175/JCLI3366.1

Indices and tables

• Index

• Module Index

• Search Page