Bias Adjustment and Downscaling Algorithms¶
The xsdba submodule provides a collection of bias-adjustment methods meant to correct for systematic biases found in climate model simulations relative to observations. Almost all adjustment algorithms conform to the train - adjust scheme, meaning that adjustment factors are first estimated on training data sets, then applied in a distinct step to the data to be adjusted. Given a reference time series (ref), historical simulations (hist) and simulations to be adjusted (sim), any bias-adjustment method would be applied by first estimating the adjustment factors between the historical simulation and the observation series, and then applying these factors to sim`, which could be a future simulation:
# Create the adjustment object by training it with reference and model data, plus certain arguments
Adj = Adjustment.train(ref, hist, group="time.month")
# Get a scenario by applying the adjustment to a simulated timeseries.
scen = Adj.adjust(sim, interp="linear")
Adj.ds.af # adjustment factors.
Most method support both additive and multiplicative correction factors. Also, the group argument allows adjustment factors to be estimated independently for different periods: the full time series, months, seasons or day of the year. For monthly groupings, the interp argument then allows for interpolation between adjustment factors to avoid discontinuities in the bias-adjusted series. See Grouping below.
The same interpolation principle is also used for quantiles. Indeed, for methods extracting adjustment factors by quantile, interpolation is also done between quantiles. This can help reduce discontinuities in the adjusted time series, and possibly reduce the number of quantile bins that needs to be used.
Modular Approach¶
The module attempts to adopt a modular approach instead of implementing published and named methods directly. A generic bias adjustment process is laid out as follows:
preprocessing on ref, hist and sim (using methods in
xsdba.processingorxsdba.detrending)creating and training the adjustment object
Adj = Adjustment.train(obs, sim, **kwargs)(fromxsdba.adjustment)adjustment
scen = Adj.adjust(sim, **kwargs)post-processing on scen (for example: re-trending)
The train-adjust approach allows to inspect the trained adjustment object. The training information is stored in the underlying Adj.ds dataset and usually has a af variable with the adjustment factors. Its layout and the other available variables vary between the different algorithm, refer to Adjustment methods.
Parameters needed by the training and the adjustment are saved to the Adj.ds dataset as a adj_params attribute.
Parameters passed to the adjust call are written to the history attribute in the output scenario DataArray.
Grouping¶
For basic time period grouping (months, day of year, season), passing a string to the methods needing it is sufficient.
Most methods acting on grouped data also accept a window int argument to pad the groups with data from adjacent ones.
Units of window are the sampling frequency of the main grouping dimension (usually time). For more complex grouping,
one can pass an instance of xsdba.base.Grouper directly. For example, if one wants to compute the factors
for each day of the year but across all realizations of an ensemble : group = Grouper("time.dayofyear", add_dims=['realization']).
In a conventional empirical quantile mapping (EQM), this will compute the quantiles for each day of year and all realizations together, yielding a single set of adjustment factors for all realizations.
Warning
If grouping according to the day of the year is needed, the xarray.DataArray.convert_calendar is useful
tools to manage the different calendars that the input data can have. By default, if two different calendars are
passed in xsdba, the adjustment factors will always be interpolated to the largest range of day of the years but this can
lead to strange values, so we recommend converting the data beforehand to a common calendar.
Application in multivariate settings (climate science examples)¶
When applying univariate adjustment methods to multiple variables, some strategies are recommended to avoid introducing unrealistic artifacts in adjusted outputs.
Minimum and maximum temperature¶
When adjusting both minimum and maximum temperature, adjustment factors sometimes yield minimum temperatures larger than the maximum temperature on the same day, which of course, is nonsensical. One way to avoid this is to first adjust maximum temperature using an additive adjustment, then adjust the diurnal temperature range (DTR) using a multiplicative adjustment, and then determine minimum temperature by subtracting DTR from the maximum temperature [Agbazo and Grenier, 2020, Thrasher et al., 2012].
Relative and specific humidity¶
When adjusting both relative and specific humidity, we want to preserve the relationship between both. To do this, Grenier [2018] suggests to first adjust the relative humidity using a multiplicative factor, ensure values are within 0-100%, then apply an additive adjustment factor to the surface pressure before estimating the specific humidity from thermodynamic relationships.
Radiation and precipitation¶
In theory, short wave radiation should be capped when precipitation is not zero, but there is as of yet no mechanism proposed to do that, see Hoffmann and Rath [2012].
Usage examples¶
The usage of this module is documented in two example notebooks: Simple and advanced examples.
Experimental wrap of SBCK¶
The SBCK python package implements various bias-adjustment methods, with an emphasis on multivariate methods and with
a care for performance. If the package is correctly installed alongside xsdba, the methods will be wrapped into
xsdba.adjustment.Adjust classes (names beginning with SBCK_) with a minimal overhead so that they can
be parallelized with dask and accept xarray objects. For now, these experimental classes can’t use the train-adjust
approach, instead they only provide one method, adjust(ref, hist, sim, multi_dim=None, **kwargs) which performs all
steps : initialization of the SBCK object, training (fit) and adjusting (predict). All SBCK wrappers accept a
multi_dim argument for specifying the name of the “multivariate” dimension. This wrapping is still experimental and
some bugs or inconsistencies might exist. To see how one can install that package, see Extra Dependencies.
Notes for Developers¶
To be scalable and performant, the sdba module makes use of the special decorators xsdba.base.map_blocks()
and xsdba.base.map_groups(). However, they have the inconvenient that functions wrapped by them are unable
to manage xarray attributes (including units) correctly and their signatures are sometime wrong and often unclear. For
this reason, the module is often divided in two parts : the (decorated) compute functions in a “private” file
(ex: _adjustment.py) and the user-facing functions or objects in corresponding public file (ex: adjustment.py).
See the advanced_example notebook for more info on the reasons for this move.
Other restrictions : map_blocks will remove any “auxiliary” coordinates before calling the wrapped function and will
add them back on exit.
User API¶
See: xsdba Module
Developer API¶
See: xsdba Utilities
_xsdba-footnotes:
SDBA Footnotes¶
Médard Noukpo Agbazo and Patrick Grenier. Characterizing and avoiding physical inconsistency generated by the application of univariate quantile mapping on daily minimum and maximum temperatures over Hudson Bay. International Journal of Climatology, 40(8):3868–3884, 2020. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/joc.6432 (visited on 2022-08-03), doi:10.1002/joc.6432.
Mégane Alavoine and Patrick Grenier. The distinct problems of physical inconsistency and of multivariate bias involved in the statistical adjustment of climate simulations. International Journal of Climatology, pages 1–23, September 2022. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/joc.7878 (visited on 2022-11-16), doi:10.1002/joc.7878.
L. Baringhaus and C. Franz. On a new multivariate two-sample test. Journal of Multivariate Analysis, 88(1):190–206, January 2004. URL: https://www.sciencedirect.com/science/article/pii/S0047259X03000794 (visited on 2022-07-29), doi:10.1016/S0047-259X(03)00079-4.
Alex J. Cannon. Multivariate quantile mapping bias correction: an N-dimensional probability density function transform for climate model simulations of multiple variables. Climate Dynamics, 50(1):31–49, January 2018. URL: https://doi.org/10.1007/s00382-017-3580-6 (visited on 2022-07-29), doi:10.1007/s00382-017-3580-6.
Alex J. Cannon. MBC: Multivariate Bias Correction of Climate Model Outputs. October 2020. URL: https://CRAN.R-project.org/package=MBC (visited on 2022-07-29).
Alex J. Cannon, Stephen R. Sobie, and Trevor Q. Murdock. Bias Correction of GCM Precipitation by Quantile Mapping: How Well Do Methods Preserve Changes in Quantiles and Extremes? Journal of Climate, 28(17):6938–6959, September 2015. URL: https://journals.ametsoc.org/view/journals/clim/28/17/jcli-d-14-00754.1.xml (visited on 2022-07-29), doi:10.1175/JCLI-D-14-00754.1.
William S. Cleveland. Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association, 74(368):829–836, December 1979. URL: https://www.tandfonline.com/doi/abs/10.1080/01621459.1979.10481038 (visited on 2022-07-29), doi:10.1080/01621459.1979.10481038.
Bertrand Denis, Jean Côté, and René Laprise. Spectral Decomposition of Two-Dimensional Atmospheric Fields on Limited-Area Domains Using the Discrete Cosine Transform (DCT). Monthly Weather Review, 130(7):1812–1829, July 2002. doi:10.1175/1520-0493(2002)130<1812:SDOTDA>2.0.CO;2.
Michel Déqué. Frequency of precipitation and temperature extremes over France in an anthropogenic scenario: Model results and statistical correction according to observed values. Global and Planetary Change, 57(1):16–26, May 2007. URL: https://www.sciencedirect.com/science/article/pii/S0921818106002748 (visited on 2022-07-29), doi:10.1016/j.gloplacha.2006.11.030.
Alexandre Gramfort. LOWESS : Locally weighted regression. October 2015. URL: https://gist.github.com/agramfort/850437 (visited on 2022-08-05).
Patrick Grenier. Two Types of Physical Inconsistency to Avoid with Univariate Quantile Mapping: A Case Study over North America Concerning Relative Humidity and Its Parent Variables. Journal of Applied Meteorology and Climatology, 57(2):347–364, February 2018. URL: https://journals.ametsoc.org/view/journals/apme/57/2/jamc-d-17-0177.1.xml (visited on 2022-08-03), doi:10.1175/JAMC-D-17-0177.1.
Nicholas J. Higham. Computing a nearest symmetric positive semidefinite matrix. Linear Algebra and its Applications, 103:103–118, 1988. URL: https://www.sciencedirect.com/science/article/pii/0024379588902236, doi:https://doi.org/10.1016/0024-3795(88)90223-6.
Jan Hnilica, Martin Hanel, and Vladimír Puš. Multisite bias correction of precipitation data from regional climate models. International Journal of Climatology, 37(6):2934–2946, 2017. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/joc.4890 (visited on 2022-07-29), doi:10.1002/joc.4890.
Holger Hoffmann and Thomas Rath. Meteorologically consistent bias correction of climate time series for agricultural models. Theoretical and Applied Climatology, 110(1):129–141, October 2012. URL: https://doi.org/10.1007/s00704-012-0618-x (visited on 2022-08-03), doi:10.1007/s00704-012-0618-x.
Jonathan Jalbert. Extreme value analysis package for Julia. June 2022. URL: https://github.com/jojal5/Extremes.jl (visited on 2022-07-29).
Dirk L. Knol and Jos M.F. ten Berge. Least-squares approximation of an improper correlation matrix by a proper one. Psychometrika, 54(1):53–61, 1989. doi:10.1007/BF02294448.
Francesco Mezzadri. How to generate random matrices from the classical compact groups. February 2007. URL: https://arxiv.org/abs/math-ph/0609050 (visited on 2022-11-16), doi:10.48550/arXiv.math-ph/0609050.
F. Pitie, A.C. Kokaram, and R. Dahyot. N-dimensional probability density function transfer and its application to color transfer. In Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, volume 2, 1434–1439 Vol. 2. October 2005. doi:10.1109/ICCV.2005.166.
Y. Robin, M. Vrac, P. Naveau, and P. Yiou. Multivariate stochastic bias corrections with optimal transport. Hydrology and Earth System Sciences, 23(2):773–786, 2019. URL: https://hess.copernicus.org/articles/23/773/2019/, doi:10.5194/hess-23-773-2019.
Yoann Robin. SBCK: Statistical Bias Correction Kit. 2021. URL: https://github.com/yrobink/SBCK-python (visited on 2024-07-03).
Philippe Roy, Gabriel Rondeau-Genesse, Jonathan Jalbert, and Élyse Fournier. Climate scenarios of extreme precipitation using a combination of parametric and non-parametric bias correction methods in the province of québec. Canadian Water Resources Journal, june 2023. URL: https://www.tandfonline.com/doi/full/10.1080/07011784.2023.2220682, doi:10.1080/07011784.2023.2220682.
Philippe Roy, Trevor James Smith, Tony Kelman, Éloïse Nolet-Gravel, Elliot Saba, Fidel Thomet, Julia TagBot, and Gael Forget. JuliaClimate/ClimateTools.jl: v0.23.1. September 2021. URL: https://zenodo.org/record/5399172 (visited on 2022-07-29), doi:10.5281/zenodo.5399172.
Jürg Schmidli, Christoph Frei, and Pier Luigi Vidale. Downscaling from GCM precipitation: a benchmark for dynamical and statistical downscaling methods. International Journal of Climatology, 26(5):679–689, 2006. URL: https://onlinelibrary.wiley.com/doi/abs/10.1002/joc.1287 (visited on 2022-07-29), doi:10.1002/joc.1287.
Gabor Szekely and Maria Rizzo. Testing for equal distributions in high dimension. InterStat, November 2004.
Matthias Jakob Themeßl, Andreas Gobiet, and Georg Heinrich. Empirical-statistical downscaling and error correction of regional climate models and its impact on the climate change signal. Climatic Change, 112(2):449–468, May 2012. URL: https://doi.org/10.1007/s10584-011-0224-4 (visited on 2022-07-29), doi:10.1007/s10584-011-0224-4.
B. Thrasher, E. P. Maurer, C. McKellar, and P. B. Duffy. Technical Note: Bias correcting climate model simulated daily temperature extremes with quantile mapping. Hydrology and Earth System Sciences, 16(9):3309–3314, September 2012. Publisher: Copernicus GmbH. URL: https://hess.copernicus.org/articles/16/3309/2012/ (visited on 2022-08-03), doi:10.5194/hess-16-3309-2012.