Climatic Change DOI 10.1007/s10584-015-1412-4
Testing climate models using an impact model: what are the advantages? Marc Stéfanon 1 & Nicolas K. Martin-StPaul 1,6 & Paul Leadley 1 & Sophie Bastin 2 & Alessandro Dell’Aquila 3 & Philippe Drobinski 4 & Clemente Gallardo 5
Received: 8 April 2014 / Accepted: 19 April 2015 # Springer Science+Business Media Dordrecht 2015
Abstract Global and regional climate model (GCM and RCM) outputs are often used as climate forcing for ecological impact models, and this potentially results in large cumulative errors because information and error are passed sequentially along the modeling chain from GCM to RCM to impact model. There are also a growing number of Earth system modeling platforms in which climate and ecological models are dynamically coupled, and in this case error amplification due to feedbacks can lead to even more serious problems. It is essential in both cases to rethink the organization of evaluation which typically relies on independent validation at each successive step, and to rely more heavily on analyses that cover the full modeling chain and thus require stronger interactions between climate and impact modelers. In this paper, we illustrate the benefits of using impact models as an additional source of information for evaluating climate models. Four RCMs that are part of the HyMeX (Hydrological cycle in Mediterranean EXperiment) and Mediterranean CORDEX projects (MED-CORDEX) were tested with observed climatology and a process-based model of European beech (Fagus sylvatica L.) tree growth and forest ecosystem functioning that has been rigorously validated. This two part analysis i) indicates that evaluation of RCMs on climate variables alone may be insufficient to determine the suitability of RCMs for studies of climate-forest interactions and ii) points to areas of improvement in these RCMs that would improve impact studies or behavior in coupled climate-ecosystem models over the spatial domain studied.
* Marc Stéfanon
[email protected] 1
Laboratoire d’Écologie Systématique et Évolution (ESE), (UMR 8079 CNRS/Univ. Paris-Sud/ AgroParisTech), Orsay, France
2
Laboratoire Atmospheres, Milieux, Observations Spatiales (LATMOS), Institut Pierre Simon Laplace (CNRS/UVSQ/UPMC), Guyancourt, France
3
Ente per le Nuove Technologie, l’Energia e l’Ambiente (ENEA), Climate Section, Rome, Italy
4
Laboratoire de Météorologie Dynamique (LMD) - Institut Pierre Simon Laplace, (CNRS/Ecole Polytechnique/ENS/UPMC), Paris, France
5
Instituto de Ciencias Ambientales de la UCLM, Toledo, Spain
6
Ecologie des Forêts Mediterraneennes, INRA, UR629, F-84914 Avignon, France
Climatic Change
1 Introduction Global and regional climate model (GCM and RCM) outputs are often used as climate forcing for ecological impact models. For the ecological community, the assessment of the climatic information is a critical key step of the impact modeling process. However evaluation of climate models using climate data is a complex task for several reasons. First, there is a large number of output variables (>100), some of which are difficult to compare with data due to lack of good measurements. Second, most diagnostics examine only a single variable at a time and generally focus on testing for systematic bias and the capacity to reproduce temporal and spatial patterns for a wide range of scales (Brands et al. 2013; Flaounas et al. 2013). Finally, the different variables are intrinsically linked through complex non-linear relationships that can include thresholds. Substantial thought has been given to developing a framework to deal with the enormous amount of data produced in the last few decades by various international exercises of model inter-comparison, such as the Coupled Model Intercomparison Project (CMIP - Taylor et al. 2012) or the COordinated Downscaling EXperiment (CORDEX - Giorgi et al. 2009). Current methodologies rely on integrated indicators that summarize information (Taylor 2001, Teuling et al. 2011). Climate models are imperfect by construction and do not perform evenly for all simulated variables and/or evaluation indicators because of the many sources of uncertainty (e.g., physical parameterization, numerical scheme, boundary conditions) (Flaounas et al. 2013; Di Luca et al. 2014). Their evaluation is also made difficult because of the uncertainty of the observations (Flaounas et al. 2012), hence the importance given to inter-comparison exercises. Once these evaluations performed, a key question remains: how good is good enough? The answer depends on the application. One under-explored way in global and regional climate modeling is process-based evaluation rather than a ‘holistic’ variable-based evaluation (Lung et al. 2013). Along the different components of the modeling chain from GCM to RCM to impact model, information is passed and error is typically assessed sequentially. Rethinking the organization of this chain from linear to cyclical by using impact models as an additional source of information for evaluating climate models can provide many benefits, but the use of impact indicators to test climate models is infrequent. This approach is particularly insightful because many impacts integrate a broad range of climate signals and can be sensitive to small changes in climate drivers (Lung et al. 2013). We have tested four RCMs with both observed climatology and two impact models. Our analysis focuses on climate impacts on forests using European beech (Fagus sylvatica L.) as an example. This species is a dominant and a representative tree of temperate deciduous broadleaf forests in Europe, and is of high economic importance for timber (Fang and Lechowicz 2006). The long time period between planting and harvesting of trees in exploited forests means that beech trees often integrate the local climatic signals over periods exceeding a century. Moreover the low migration capacity and long life cycle of trees in non-managed forests make trees and forests particularly vulnerable to climate change impacts (Lindner et al. 2010; Cheaib et al. 2012). We argue that analyzing the differences between these two diagnostics highlights i) the non-linear error propagation along the modeling chain, ii) the weighting of climatic processes and variables of interest for a given type of impact model, and iii) the variables and processes of the climate model requiring improvement.
Climatic Change
2 Methodology 2.1 Models and data Five different climate datasets over France were used: four are provided by RCMs and one is a reference dataset based on surface meteorological observations. The RCM outputs were provided by the HyMeX (Hydrological cycle in Mediterranean EXperiment, Drobinski et al. 2014) and Mediterranean CORDEX (MED-CORDEX, Ruti et al. 2014) archives: ALADIN (Colin et al. 2010), RegCM (Pal et al. 2007), WRF (Skamarock et al. 2008), PROMES (Domínguez et al. 2010). These models were selected based on the availability of the daily data required by the tree growth and forest ecosystem model, CASTANEA (see below). The RCM simulations were performed at a 50 km resolution for the period from 1989 to 2008 by a dynamical downscaling of the ERA-INTERIM reanalysis (Dee et al. 2011). The SAFRAN analysis (Quintana-Seguí et al. 2008) was used as a reference dataset. SAFRAN is derived from the climatological network of Météo-France and provided the required variables on a 8 km grid. We used two models of tree response to climate, the process-based tree growth and forest ecosystem model CASTANEA (Dufrêne et al 2005) and the niche-based model BIOMOD (Thuiller 2003), as the basis for assessing the performance of atmospheric variables provided by the four RCMs. CASTANEA is a forest soil-vegetation-atmosphere model coupled with a tree growth module. It simulates carbon and water fluxes at a daily time step for an average tree in a homogeneous stand of forest using 5 atmospheric variables (temperature, rainfall, solar downward radiation, wind speed at 10 m, relative humidity at 2 m) provided by forcing datasets. CASTANEA simulates carbon and water fluxes, including gross and net ecosystem photosynthesis, respiration, transpiration, latent heat flux, soil water content and tree growth for several major European tree species including European beech (Davi et al. 2005, 2008; Delpierre et al. 2012). CASTANEA has been thoroughly validated for beech and several other major European tree species using ecosystem CO2 flux, water flux and tree growth data from multiple sites (Davi et al. 2005, 2008; Cheaib et al. 2012; Delpierre et al. 2012). CASTANEA can also be used to simulate the distributional range (presence/absence) of a tree species by assuming that tree growth is a proxy of its ability to persist in a given environment (Cheaib et al. 2012). It has been validated against current tree distribution using the French National Forest Inventory (NFI, for more information see www.ifn.fr). BIOMOD is a correlative species distribution model which links observed presences of a species to climate variables trough statistical relationships. This type of model is used widely in the ecological community with around 22,000 related publications over the 1999–2009 period (Thuiller et al. 2009). These models require mean annual climate variables as input (e.g., annual precipitation, temperature, potential evapotranspiration).
2.2 RCM evaluation RCMs were evaluated using two approaches: i) a conventional method using standard tools for evaluation based on climatic variables (Taylor diagrams and overall bias calculations) and ii) an ecological approach based on the simulation of tree distributional range computed with the growth model CASTANEA and a correlative species distribution model. Temperature, rainfall and relative humidity were used in the climatic evaluation based on knowledge that these variables play the most important roles in simulating tree growth and
Climatic Change
distribution in CASTANEA (Davi et al. 2005, 2008; Cheaib et al. 2012; Delpierre et al. 2012). Climate variables were computed over the period May-August since most of annual photosynthesis and radial growth of beech occurs during this time (Lebaube et al. 2000; Schmitt et al. 2000). Multi-annual lags could potentially add a confounding factor in this analysis, but observations from the French Permanent Plot Network (RENECOFOR - Lebourgeois et al. 2005) suggest that the role of weather and drought conditions during the previous season are of little relevance for growth dynamics of European beech. The climatic evaluation was performed for each climatic variable using Taylor diagrams for each model. Taylor diagrams provide a graphical summary of how closely a given simulated variable matches a reference variable and are a standard method of model evaluation in the climate modeling community (Taylor 2001). The similarity between the two patterns is quantified using correlation, centered root-meansquare difference and the magnitude of their variations (standard deviations). Taylor diagrams put a heavy emphasis on the ability of models to reproduce spatial and temporal patterns. Taylor diagrams do not provide information about overall bias because the means of the fields are subtracted out before computing the secondorder statistics used in the diagram (Taylor 2001). Spatially and temporally average overall biases were also calculated for all three climate variables. Taylor diagrams are widely used by climate community to evaluate model performance. Other performance metrics exist and could provide additional insight into the evaluation of climatic variables (Gleckler et al. 2008). For the ecological evaluation, the RCM climatic outputs as well as the SAFRAN reference dataset were used as inputs for CASTANEA to stimulate the European beech growth. CASTANEA simulations were performed for the study area (France) on a 8 km grid to match the soil water holding capacity database (Cheaib et al. 2012). Consequently, RCM climatic inputs at 50 km were resampled on an 8 km grid. Because of highly non-linear effect linked to soil moisture, we consider that the loss of information by upscaling to 50 km the soil database is more critical for the results than a rescaling of climatic data to 8 km. Tree growth (gC/m2/yr) was transformed to presence/absence by comparing the map of simulations to the binary map of presence/ absence observations. This was achieved by computing a threshold value of tree growth that maximizes the goodness-of-fit of the CASTANEA simulations forced by SAFRAN. Goodness-of-fit is calculated with the TSS (True Skill Statistics, Allouche et al. 2006), a statistic ranking that has values ranging between −1 and 1. TSS accounts for both the presences correctly predicted and the absences correctly predicted. A score of 1 means that both presence and absence are predicted perfectly whereas a score of −1 means that no points of the grid are predicted correctly. Simulations have also been performed with an ensemble average of five different statistical species distribution models implemented in the BIOMOD modeling platform (Thuiller 2014). Statistical relationships were calibrated using a sub-sample of the observed species distribution with the SAFRAN dataset and evaluated using the remaining data. BIOMOD has been extensively evaluated over France and generally provides better results than other correlative species distribution models because it is based on a multi-model ensemble method (Marmion et al. 2009; Cheaib et al. 2012). The same calibration was applied to others simulations forced by the RCMs and goodness-of-fit indices (TSS) were computed. We used SAFRAN for the calibration since it is our climatic reference.
Climatic Change
2.3 Evaluation of the individual climate variables for the WRF RCM Climate models cannot represent perfectly all aspects of climate; as a result errors in the temporal pattern, the spatial patterns as well as systematic bias are often reported. Such biases often preclude the dynamic coupling between vegetation models and climate models and the correct assessment of impacts unless some form of bias correction is performed (Ehret et al. 2012; Ruffault et al. 2014). Based on the RCM evaluations described in subsection 2.2), we performed a sensitivity analysis with the WRF model in order to identify which climate variables contribute the most to deteriorate the ecological signal produced by CASTANEA. To do so, five sensitivity tests are carried out by using different climate datasets. Each test includes one variable from WRF while other variables are provided by SAFRAN.
3 Results and discussion 3.1 Evaluation of RCMs based on climatic variables Figure 1 shows the Taylor diagrams for the temporal and spatial patterns of surface air temperature, rainfall, and surface relative humidity, with respect to the SAFRAN analysis. Table 1 shows the biases of each variable over the period May-August. The temporal variability of temperature is well reproduced by all RCMs since all points are tightly clustered around the SAFRAN reference point in the Taylor diagram. The spatial pattern of temperature is reasonably well reproduced by three of the RCMs (correlation coefficient of ca. 0.7 and standard deviations that are similar the SAFRAN reference), but RegCM has much a lower correlation coefficient (0.2) and underestimates spatial variability. The RMSD for spatial variability is near or above 2 °C for all models indicating substantial spatial heterogeneity in the relationship between the models and the reference. Systematic bias is less than 1 °C for all the models except WRF which shows a large warm bias during the summer (+2.4 °C, Table 1). Compared to temperature, temporal correlations of rainfall between RCMs and the SAFR AN reference were lower (ca. 0.7–0.9). The ability of PROMES to simulate temporal variation in rainfall is slightly lower than for the other RCMs based on correlation and RMSD. Concerning the spatial pattern of rainfall, RegCM has the lowest correlation with the SAFR AN analysis (0.4), when compared to others models (ca. 0.7). ALADIN and PROMES have higher spatial variability (standard deviation) and RegCM has lower spatial variability than the reference. In terms of total rainfall during the growing season (May–August), PROMES, RegCM and ALADIN are close to SAFRAN (differences of
