PUBLICATIONS Journal of Advances in Modeling Earth Systems RESEARCH ARTICLE 10.1002/2013MS000298 Key Points: FME was fully wrapped into EDCM to support sensitivity/uncertainty analysis FME functions can help identify the relationship between parameter and output Parameter uncertainty has distinct effects for variables in different seasons
Correspondence to: Y. Wu,
[email protected] S. Liu,
[email protected] Citation: Wu, Y., S. Liu, Z. Huang, and W. Yan (2014), Parameter optimization, sensitivity, and uncertainty analysis of an ecosystem model at a forest flux tower site in the United States. J. Adv. Model. Earth Syst., 6, 405–419, doi:10.1002/2013MS000298. Received 18 DEC 2013 Accepted 22 APR 2014 Accepted article online 26 APR 2014 Published online 27 MAY 2014
Parameter optimization, sensitivity, and uncertainty analysis of an ecosystem model at a forest flux tower site in the United States Yiping Wu1, Shuguang Liu2,3, Zhihong Huang4, and Wende Yan4 1
ASRC Research and Technology Solutions, U.S. Geological Survey Earth Resources Observation and Science Center, Sioux Falls, South Dakota, USA, 2U.S. Geological Survey Earth Resources Observation and Science Center, Sioux Falls, South Dakota, USA, 3Geographic Information Science Center of Excellence, South Dakota State University, Brookings, South Dakota, USA, 4National Engineering Laboratory for Applied Technology of Forestry and Ecology in Southern China, Central South University of Forestry and Technology, Changsha, China
Abstract Ecosystem models are useful tools for understanding ecological processes and for sustainable management of resources. In biogeochemical field, numerical models have been widely used for investigating carbon dynamics under global changes from site to regional and global scales. However, it is still challenging to optimize parameters and estimate parameterization uncertainty for complex process-based models such as the Erosion Deposition Carbon Model (EDCM), a modified version of CENTURY, that consider carbon, water, and nutrient cycles of ecosystems. This study was designed to conduct the parameter identifiability, optimization, sensitivity, and uncertainty analysis of EDCM using our developed EDCM-Auto, which incorporated a comprehensive R package—Flexible Modeling Framework (FME) and the Shuffled Complex Evolution (SCE) algorithm. Using a forest flux tower site as a case study, we implemented a comprehensive modeling analysis involving nine parameters and four target variables (carbon and water fluxes) with their corresponding measurements based on the eddy covariance technique. The local sensitivity analysis shows that the plant production-related parameters (e.g., PPDF1 and PRDX) are most sensitive to the model cost function. Both SCE and FME are comparable and performed well in deriving the optimal parameter set with satisfactory simulations of target variables. Global sensitivity and uncertainty analysis indicate that the parameter uncertainty and the resulting output uncertainty can be quantified, and that the magnitude of parameter-uncertainty effects depends on variables and seasons. This study also demonstrates that using the cutting-edge R functions such as FME can be feasible and attractive for conducting comprehensive parameter analysis for ecosystem modeling.
1. Introduction The terrestrial biosphere, particularly forest ecosystem, plays a pivotal role in the global carbon cycle and partially mitigates the rising atmospheric carbon dioxide (CO2) concentration due to its role as a carbon sink [Schimel, 1995; Gurney et al., 2002; IPCC, 2007; Heimann and Reichstein, 2008; Le Quere et al., 2013]. Understanding the carbon dynamics of an ecosystem is fundamentally important for investigating the driving forces and mechanism of carbon sequestration and for regional ecosystem planning and management as well [Miehle et al., 2006; Zhu et al., 2010]. Process-based ecosystem models are useful tools for carbon cycling research because they incorporate a representation of the simulated natural systems [Williams et al., 2005], thus they can help understand and quantify terrestrial carbon exchanges and simulate components that are difficult to measure [Fatichi et al., 2012a, 2012b]. Models can also be used to integrate the local measurements of ecological processes and scale up to regional predictions [Hurtt et al., 2002; Luo et al., 2008; Xia et al., 2013; Hashimoto et al., 2013; Wang et al., 2011; Horn and Schulz, 2011a, 2011b]. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
WU ET AL.
Although these process-based models are used to represent the natural systems and examine the dynamics of carbon, water, and other elements, they usually contain some parameters that need to be calibrated by model inversion because of the lack of field measurements, mismatch between measurement and modeling scales, and heterogeneity of the physical environment for regional modeling [Beven, 2001; Nandakumar and Mein, 1997; Foglia et al., 2009]. This procedure (model calibration) using observation data is usually necessary for numerical models. For one thing, inverse modeling with required observation data can help derive an
C 2014. The Authors. V
405
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
optimal parameter set using the manual trial-and-error approach involving user’s knowledge and subjective judgment (i.e., manual calibration) or an objective optimization algorithm (i.e., automatic calibration) [Gupta et al., 1999]. The former is labor intensive and tends to get stuck in a local optimum. In contrast, the latter can initiate hundreds or thousands of model iterations for deriving the globally optimal results, but it requires additional programming efforts to develop or introduce a proper optimization algorithm for an existing model. Additionally, a single best parameter set can only provide a deterministic model simulation. The existence of parameter uncertainty will lead to uncertainties in model output (e.g., carbon fluxes). Therefore, a proper sensitivity and uncertainty analysis is preferable because it produces model simulations with a confidence range (band) instead of specific output [Beven, 2001; Van Griensven et al., 2008; Ng et al., 2010]. In this sense, it is critical to investigate the parameter uncertainty and quantify its effects on the model output especially for large complex ecological models [Keenan et al., 2011; Wang et al., 2009; Richardson et al., 2010]. A data assimilation technique is a key tool to improve ecological predictions and estimate uncertainties based on case studies [Luo et al., 2011; Medlyn et al., 2005]. For example, Tang and Zhuang [2008, 2009] found that ensemble process-based biogeochemistry model simulations with Bayesian inference techniques could provide more serious estimates of carbon dynamics and associated uncertainties. From a literature review and our own implementations, a number of methods have been widely used for environmental models to conduct parameter optimization, and sensitivity and uncertainty analysis. Shuffled Complex Evolution (SCE) [Duan et al., 1992] and pseudorandom search [Price, 1977] algorithms have been used for parameter optimization and Metropolis-Hastings (MH) [Hastings, 1970] and delayed rejection and adaptive Metropolis (DRAM) [Haario et al., 2006] algorithms have been used for Markov Chain Monte Carlo analysis. In particular, the R package Flexible Modeling Environment (FME) [Soetaert and Petzoldt, 2010] was promising because it is open source and can implement a variety of functionalities. The objective of this study is to evaluate the carbon dynamics of an ecosystem using the well-established EDCM model and conduct a comprehensive modeling analysis with a focus on parameter local sensitivity and identifiability analyses, multiparameter and multiobjective (four key variables) model inversion, and parameter global sensitivity and uncertainty analysis using the R package FME. For this purpose, we used a mature deciduous forest flux tower site—the Harvard Forest in Massachusetts in the United States—as a case study to take advantage of the continuous and multiple measurements of carbon and water fluxes based on the eddy covariance technique and biometrics including Gross Primary Production (GPP), Net Ecosystem Exchange (NEE), Evapotranspiration (ET), and Leaf Area Index (LAI).
2. Materials and Methods 2.1. EDCM Description EDCM is a process-based biogeochemical model used to simulate carbon and nitrogen cycles in diverse ecosystems at a monthly time step and take into account the impacts of land management and disturbances [Tan et al., 2009; Liu et al., 2003]. EDCM [Liu et al., 2003] is a modified version of CENTURY (version IV) [Parton et al., 1987, 1994], but EDCM uses up to 10 soil layers to simulate the soil organic carbon (SOC) dynamics in the whole soil profile, instead of the one single top-layer structure of CENTURY. EDCM can dynamically keep track of the evolution of the soil profile and carbon storage as influenced by soil erosion and deposition [Liu et al., 2003]. In brief, like CENTURY, EDCM includes three soil organic matter (SOM) pools (active, slow, and passive) with different decomposition rates, aboveground and belowground litter pools, and a surface microbial pool that is associated with decomposing surface litter [USDA, 1993]. The plant production module has pools for live shoots and roots including leaves, fine branches, large woods, fine roots, and coarse roots. The potential plant production is a function of a genetic maximum defined for each plant (i.e., parameter PRDX) and 0–1 scalars depending on soil temperature, moisture status, and nutrient supplies. This carbon pool model focuses on tracking the dynamics of carbon storage in each pool. Driven by its interface—the General Ensemble Modeling System (GEMS), EDCM has been used to assess the carbon stocks and fluxes under changing climate and land covers for the baseline and projection periods across the conterminous United States [Liu et al., 2012b; Zhu, 2011]. 2.2. The Functions of FME Adopted in This Study The R package FME [Soetaert and Petzoldt, 2010] was designed to conduct comprehensive analysis, which includes algorithms for inverse modeling with a group of optimization methods, parameter identifiability
WU ET AL.
C 2014. The Authors. V
406
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
analysis, and sensitivity and Monte Carlo analysis. FME implements part of the functions from a previous distribution—Fortran environment FEMME [Soetaert et al., 2002]. Detailed information about the functions of FME is available from the R Archive Network at http://CRAN.R-project.org/package5FME. Here we briefly list a few major functions of FME used in our study: 1. sensFun: estimates the local sensitivity of the model output to the parameter values, and it works by measuring the model output change responding to a small change of a single parameter at one time (model run) [Brun et al., 2001; Soetaert and Herman, 2009] as shown below:
Si;j 5
@yj Vpi @pi Vyj
(1)
where Si,j is the normalized and dimensionless sensitivity of parameter i for model variable j, yj is a model output variable (e.g., GPP), pi is a parameter (e.g., PRDX), Vyj is the scaling of variable yj and usually equal to the variable value, and Vpi is the scaling of parameter pi and usually equal to the parameter value. 2. Collin: uses the above local sensitivity results (the normalized and dimensionless sensitivities) as input to estimate the approximate linear dependence (‘‘collinearity’’) of all possible parameter sets with the approach described in Brun et al. [2001], and its computing equation is shown below:
1 c5 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi T ^ min ðEV½S^ SÞ
(2)
where ^S contains the columns of the sensitivity matrix that correspond to the parameters included in the set, and EV estimates the eigenvalues. 3. modFit: finds the best fit parameters for nonlinear model-data fitting, and it embraces a group of methods including R’s built-in minimization routines (optim, nls, and nlminb), nls.lm from package minpack.lm [Elzhov and Mullen, 2009], and the pseudorandom search algorithm (denoted as PseudoOptim) of Price [Price, 1977, Soetaert and Herman, 2009]. The last algorithm was selected and its details are presented in the next section (section 2.3). 4. modMCMC: implements an MCMC (Bayesian analysis) method to derive the data-dependent probability distribution of the parameters. This function uses the Delayed Rejection and Adaptive Metropolis (DRAM) procedure [Haario et al., 2006; Laine, 2008], and its details are given in the next section (section 2.3). 5. sensRange: estimates the effect of parameter uncertainty on model output. This function performs a global sensitivity analysis that determines the effect on model outcome as a function of an appropriate parameter probability density function [Brun et al., 2001; Soetaert and Herman, 2009]. 6. modCRL: calculates the values of a single model variable as a function of the sensitivity parameters. It measures the effect of each individual parameter’s range on the mean value of a single model output [Soetaert and Petzoldt, 2010].
2.3. The Existing Modeling Framework and its Expansion In a previous study, we developed a generic autocalibration package (EDCM-Auto) by incorporating both SCE and R-based FME into the EDCM model for site/regional model calibration [Wu et al., 2014]. The technique details about how to convert a Fortran-based model (EDCM) into an R function and thus link it with an R package (FME) can be found in our previous studies [Wu and Liu, 2014, 2012; Wu et al., 2014]. The SCE is a heuristic global optimization algorithm, which combines the direct search method of the simplex procedure with the concept of a controlled random search [Nelder and Mead, 1965], a systematic evolution of points in the direction of global improvement, competitive evolution [Holland, 1975], and the concept of complex shuffling [Green and van Griensven, 2008; Duan et al., 1992]. This method starts with the initial selection of a ‘‘population’’ of points, randomly distributed throughout the allowable space for n parameters. The population is then partitioned into several ‘‘complexes’’ that consist of 2n 1 1 points, and each
WU ET AL.
C 2014. The Authors. V
407
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
complex evolves independently in a way based on the downhill simplex algorithm. These complexes are shuffled periodically to form new complexes in order to share the gained information. The evolution and shuffling steps are repeated until prescribed convergence criteria are satisfied [Green and van Griensven, 2008; Brath et al., 2004]. SCE conducts an efficient and robust search over the whole parameter space and finds the global optimum; SCE has been widely used for calibrating various numerical models [Duan et al., 1992; Eckhardt and Arnold, 2001; Le Ngo et al., 2007; Madsen, 2003]. R-based FME is another option for model calibration in the existing EDCM-Auto framework. Although a few optimization algorithms are available in FME, the pseudorandom search algorithm (denoted as PseudoOptim) was selected because it may be efficient for difficult problems [Soetaert and Herman, 2009] and has been successfully tested [Wu and Liu, 2012, 2014; Wu et al., 2014]. In brief, this algorithm starts with a ‘‘population’’ of parameter sets. During each calibration step, three-parameter sets are randomly selected from the population, their centroid value, and a value mirrored over a fourth random parameter can help generate a new parameter set. Then the model cost (model-data residuals) is calculated for this new parameter set. If the obtained model cost value is lower than the worst cost value in the population, the new parameter vector replaces the worst parameter vector, or it is discarded. These procedures are repeated until a requested number of runs have been performed. During this process, the average cost function of a population decreases in time as better parameter values replaces worse values [Soetaert and Herman, 2009; Price, 1977]. In brief, both SCE and FME (pseudorandom search) drive the model iteratively while searching parameter values in their allowable ranges to minimize the model cost, but they use different implementation schemes. SCE partitions the population into several ‘‘complexes’’ with each complex evolving independently based on simplex algorithm, followed by periodical shuffling to form new complexes; while FME’s pseudorandom search uses the centroid value of three randomly searched parameter sets as the mirror to generate a new parameter set based on a fourth random parameter set without a shuffling procedure. The cost function of the two methods takes the sum of squared residuals (SSR) and uses weighting factors to combine multivariable model-data residuals into a single objective function and to make the residuals dimensionless for accomplishing ‘‘multiobjective’’ calibration. Although there is a convergence criterion (i.e., 1% change of model cost), users can still set the maximum number of iterations (e.g., 3000 in this case study) to avoid overuse of computing resources. In the current study, we extended the functionalities of this package by introducing the other FME functions for parameter identifiability analysis and sensitivity and uncertainty analysis (listed in section 2.2). As a key part of the current study, the parameter uncertainty was investigated using MCMC, which works by drawing samples from a probability distribution and generating a sequence of sample values iteratively. At each iteration, MCMC picks a candidate for the next sample value based on the current value, but the candidate sample is either accepted (the candidate value will be used in the next iteration) or rejected (the candidate value is discarded and the current value is reused in the next iteration). The probability of acceptance is determined by comparing the likelihoods of the current and candidate sample values with respect to the desired distribution [Andrieu et al., 2003; Radford, 1993; Rubinstein and Kroese, 2007]. With the support of function modMCMC in FME, MCMC was implemented using the Delayed Rejection and Adaptive Metropolis (DRAM) procedure [Haario et al., 2006; Laine, 2008], which is a combination of two ideas—‘‘Delayed Rejection (DR)’’ and ‘‘Adaptive Metropolis (AM)’’—for improving the efficiency of Metropolis-Hastings (MH) type MCMC algorithm [Metropolis et al., 1953; Hastings, 1970]. ‘‘DR’’ can improve efficiency because, instead of advancing time and retaining the same position, it proposes a second stage upon rejection in MH, and a hierarchy between kernels can be exploited so that kernels that are easier to compute (in terms of CPU time) are tried first, thus saving in terms of simulation time. ‘‘AM’’ improves efficiency because it learns from the information obtained during the run of the chain, and thus based on that to accomplish efficient tuning of proposals [Haario et al., 2006]. In brief, the MCMC implementation drives a model to run repeatedly for exploring the probability distribution describing the uncertainties in model parameters and predictions [Haario et al., 2006; Laine, 2008]; the DRAM procedure (adopted in the current study) was built especially to work on high dimensional and nonlinear problems (dynamic models) due to its high efficiency [Malve et al., 2007; Soetaert and Petzoldt, 2010]. As a result, an ensemble of parameter values that represent the parameter distribution can be obtained. Finally, the updated package (EDCM-Auto) contains two options, SCE and the pseudorandom search algorithm of FME, for supporting multiparameter and ‘‘multiobjective’’ model calibration. Herein, the ‘‘multiobjective’’ calibration refers to combining multivariable model-data residuals into a single objective function
WU ET AL.
C 2014. The Authors. V
408
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
(as stated previously). Using FME can provide extra functionalities such as parameter identifiability analysis, and local/global sensitivity and MCMC analysis with instant visualization. In addition, although we used nine parameters and four target variables in the case study (see sections 2.4 and 2.5), the EDCM-Auto has the flexibility to add more parameters of interest and more target variables if their observations are available. 2.4. Study Site and Data We used the Harvard Forest Environmental Monitoring Site (HFEMS) as the case study, which is located on the Prospect Hill tract of the Harvard Forest (42 320 N, 72 100 W, elevation 340 m), near Petersham, Massachusetts, in the United States [Goulden et al., 1996; Wofsy et al., 1993; Curtis et al., 2002]. The vegetation is dominated by red oak (Quercus rubra L.) and red maple (Acer rubrum L.) with a scatter of black birch (Betula lenta L.) and eastern hemlock (Tsuga canadensis L.) [Oliver and Stephens, 1977; Hibbs, 1983; Curtis et al., 2002]. The forest stand age was 75–110 years old, with a canopy height of 20–24 m [Goulden et al., 1996]. The dominant soil types are stony to sandy loams formed from glacial till and they are classified as Typic Dystrochrepts according to soil pit descriptions [Aber et al., 1993; Magill et al., 2004]. Soil texture data from the Ameriflux website indicate the soil composition to be 66% sand, 29% silt, and 5% clay, with a bulk density of 0.9 g/cm3. The initial carbon pools (e.g., soil carbon and forest biomass) were from the national data layers of soil carbon (based on soil data from the Soil Survey Geographic database, SSURGO) and forest biomass (FIA unit and age-based biomass survey data) the Land Carbon Team built for 1992 [Zhu, 2011; Zhu et al., 2010], the starting year for the national assessment [Zhu, 2011; Zhu and Reed, 2012], and the current site simulation as well. In this study area, annual average precipitation is about 1120 mm distributed relatively evenly throughout the year [Magill et al., 2004], and the annual average air temperatures is about 7.1 C [Curtis et al., 2002], ranging from an average minimum of 212 C in January to an average maximum of 19 C in July [Magill et al., 2004]. Monthly precipitation and air temperature data were from the (Parameter-elevation Regressions on Independent Slopes Model (PRISM)) PRISM Climate Group [2012]. Nitrogen deposition to the forest was estimated at about 0.8 g/m2/yr [Ollinger et al., 1993]. Eddy covariance-based measured NEE and ET and derived GPP data are available from the Ameriflux website and the monthly data derived from the level 4 data were used in our study. The ground-based biometric measurement of LAI has one or two (occasionally) data points available for each month from spring to fall. Monthly aggregation based on this data set may have a bias especially when the LAI measurement was conducted in the early stage of a month. Nonetheless, we kept these values because they may still serve as one auxiliary measurement (at least as reference values) to help constrain and evaluate the model. In addition, the maximum value (5.6) of the LAI data series was taken as the value for parameter MAXLAI (maximum LAI achieved in a mature forest) of EDCM, and thus this parameter was not involved in the inverse modeling. In addition, we partitioned the 10 year simulation period (2001–2010) into a 6 year (2001–2006) calibration and a 4 year (2007–2010) validation periods considering the measurement of LAI is available since 2005. Thus, a 6 year calibration scheme would involve more observation data points (12 in total) than a 5 year calibration scheme (four data points only) without substantially compromising the data length for validation. 2.5. Target Variables and Parameters of Interest For this case study with the Harvard Forest flux tower site, we used four important model outputs as the target variables: GPP, Net Ecosystem Carbon Balance (NECB; the net C accumulation rate; positive values for a net sink and negative values for a net source), ET, and LAI. The corresponding observation data for these four variables are required for our modeling analysis and the data sources were stated in section 2.4. It is noted we used opposite NEE as the observed NECB data to calibrate the model and evaluate its performance. In fact, these two terms are different: (1) they have opposite signs because a positive NEE is a positive flux to the atmosphere and a positive NECB is a positive flux to the ecosystem; (2) NEE measures the vertical net inorganic C (CO2) flux between the ecosystem and the free atmosphere, but NECB involves the vertical and lateral fluxes of organic C and nonCO2 biological products (e.g., CO and CH4), and nonbiological oxidations (e.g., fire) [Chapin et al., 2006]. Nonetheless, NEE closely matches NECB in some ecosystems with homogeneous environments and low lateral fluxes (e.g., a flux tower site), but a substantial difference is
WU ET AL.
C 2014. The Authors. V
409
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
Table 1. Calibrated Parameter Values Using Shuffled Complex Evolution (SCE) and Flexible Modeling Environment (FME) Calibrated Value Parameter PRDX FWLOSS4 LEAFDR9 PPDF1 DEC4 DEC5 TEFF1 TEFF2 TEFF3 a
Description 2
Potential gross production (g C/m ) Scaling factor for potential evapotranspiration Leaf death rate in September Optimal temperature for production ( C) Maximum decomposition rate of soil organic matter with slow turnover Maximum decomposition rate of soil organic matter with intermediate turnover Temperature effect on soil decomposition rate (intercept) Temperature effect on soil decomposition rate (slope) Temperature effect on soil decomposition rate (exponent value)
Range
Initial Valuea
SCE
FME
800–1500 0.5–0.99 0.3–0.8 28–30 0.002–0.008
1150 0.75 0.55 29 0.005
1113.90 0.889 0.509 28.096 0.004
969.99 0.890 0.444 28.015 0.008
0.16–0.24
0.2
0.186
0.240
0–0.1 0.01–0.3 0.01–0.3
0.05 0.15 0.15
0.046 0.277 0.144
0.053 0.300 0.100
Initial parameter values were taken as the mean of their lower and upper bounds.
expected for wetlands, streams, estuaries, and arctic terrestrial ecosystems [Kling et al., 1991; Chapin et al., 2006; Howarth et al., 1996; Richey et al., 2002]. Through a literature review and our experience [Scheller et al., 2011; Liu et al., 2012a], we selected nine parameters, as listed in Table 1, for this case study. In particular, the potential gross production (PRDX) of a given ecosystem has both genetic and environmental components and is thus the foremost parameter used to calibrate the plant production for different species, environments, and varieties [USDA, 1993]. Brief descriptions of each parameter are given in Table 1, and further details about them can be found in the manual of the CENTURY model [USDA, 1993]. 2.6. Criteria to Assess Model Performance To evaluate the model performance with the optimal parameters derived by the model fitting algorithm, a group of criteria were used, including Percent Bias (PB), Nash-Sutcliffe efficiency (NSE) [Nash and Sutcliffe, 1970], r2 (squared correlation coefficient) [Krause et al., 2005], and Root-Mean-Square Error (RMSE)—observation Standard deviation Ratio (RSR), which is the ratio of RMSE to observation standard deviation [Singh et al., 2005; Moriasi et al., 2007]. The equations for calculating these four criteria are given in supporting information.
3. Results and Discussion In this study, we considered multiple parameters and target variables (as stated in section 2.5) to conduct the local sensitivity and parameter identifiability analysis, parameter optimization, and global sensitivity and uncertainty analysis. The results are described and discussed in the following subsections. 3.1. Local Sensitivity and Parameter Identifiability Analyses We estimated the local sensitivities of the nine parameters using the sensFun function (see section 2.2) with the initial parameter values and allowable parameter ranges (Table 1) and a relative change (i.e., an increase of 10% of the range based on the initial value) as the input. The weighted model cost (i.e., sums of squared residuals and considering standard deviation of the observation data for each variable) was also involved in this procedure. The summary statistics of the normalized and dimensionless parameter sensitivity matrix are provided in Table 2, including L1 and L2 norm, mean, minimum, maximum, and ranking. A parameter with a high absolute sensitivity value indicates that it is relatively more important or sensitive than other parameters. However, it is worth noting that sensitivity measures depend on the initial parameter values, parameter ranges, and the change interval allowed for each parameter [van Griensven, 2006]. Therefore, we usually consider both sensitivity rankings and these impacting factors as a whole. From Table 2, the parameter ranking based on L1 indicates PPDF1 is the most sensitive parameter despite the relatively narrow range, followed by PRDX, LEAFDR9, TEFF3, and FWLOSS4. The sensitivities for the other four parameters (DEC4, DEC5, TEFF1, and TEFF2) are much less than the former five in terms of the order of sensitivity magnitude
WU ET AL.
C 2014. The Authors. V
410
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
Table 2. Summary Statistics of Parameters’ Sensitivities Parameter
L1a
L2a
Mean 21
Minimum 1
Maximum 2
PRDX 1.9 6.9 3 10 1.4 22.4 3 10 1.5 3 10 3.5 3 1022 2.8 3 1021 21.2 3 1021 1.9 FWLOSS4 2.8 3 1021 21 21 21 1.2 3 10 22.1 3 10 25.0 1.7 3 101 LEAFDR9 8.3 3 10 1.9 26.7 29.2 3 101 3.1 3 102 PPDF1 1.4 3 101 4.4 3 1025 22.3 3 1026 23 3 1023 5.5 3 1023 DEC4 2.8 3 1024 2.6 3 1023 1.9 3 1023 21.3 3 1021 4.3 3 1021 DEC5 1.6 3 1022 2.4 3 1023 1.2 3 1022 24.5 3 1022 2.3 3 1021 TEFF1 1.8 3 1022 21 22 22 21 2.5 3 10 3.0 3 10 27.3 3 10 2.7 TEFF2 1.5 3 10 3.6 3 1022 21.4 3 1021 22.6 3.2 TEFF3 2.8 3 1021 X pffiffiffiffiffiffiffiffiffiffiffiffiffi a L1 5 jSi;j j=n and L2 5 Si;j 2 =n are the L1 and L2 norm, respectively, and Si,j is sensitivity of parameter i for model variable j. b The parameter sensitivity ranking is based on L1.
Rankb 2 5 3 1 9 8 7 6 4
(Table 2). We may decide to exclude these nonsensitive parameters for subsequent model fitting and uncertainty analysis to save time and computing resources, although we kept them in this case study. To examine the parameter identifiability, which is crucial for parameter estimation, we used the Collin function (see section 2.2) to assess the degree of near-linear dependence of sensitivity functions of all parameter subsets (i.e., collinearity index). Figure 1 shows the collinearity indices of all the possible parameter subsets (a total of 502 for a nine-parameter set) on all four model variables (GPP, NECB, ET, and LAI) as a whole. A collinearity index over a critical value (typically chosen to be 10–15) can assume that the corresponding parameter subset is poorly identifiable [Brun et al., 2001; Soetaert and Petzoldt, 2010; Belsley, 1991]. For this study, however, the collinearity indices for various parameter subsets were always less than or equal to six, indicating that the nine parameters we selected can be identifiable. 3.2. Parameter Optimization and Model Performance Evaluation We used SCE and FME (using the modFit function with the PseudoOptim algorithm) methods, respectively, to derive the optimal parameter values with 6 year (2001–2006) monthly observations of the four target variables (i.e., model calibration or model fitting). The subsequent 4 year (2007–2010) observations were used as the independent data to validate the model performance (i.e., model validation). There are some missing values for ET and in particular LAI. We found both SCE and FME (PseudoOptim) converged within 2700 iterations, which is reasonable considering that the five parameters are not sensitive, and TEFF1 is just around its initial value (0.5) and that TEFF2 is close to its upper limit (0.3). Our tests for model calibration indicated that a different random seed for either of the two methods can result in a different solution. We also found that the model performance for NECB is not always as good as other target variables. Therefore, we used the same random seed (1667) and set a greater weight (10) for NECB; this would help improve the simulation of NECB. The calibrated parameter values by the two methods with the same random seed and a greater Figure 1. Collinearity index of all the parameter subsets on all the four model variables: weight for NECB are listed in Gross Primary Production (GPP), Net Ecosystem Carbon Balance (NECB), EvapotranspiraTable 1. From this table, we tion (ET), and Leaf Area Index (LAI). The horizontal axis shows the number of parameters noticed that the optimized involved for possible parameter combinations—a total of 502.
WU ET AL.
C 2014. The Authors. V
411
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
Figure 2. Monthly time series of observed and simulated Gross Primary Production (GPP), Net Ecosystem Carbon Balance (NECB), Evapotranspiration (ET), and Leaf Area Index (LAI) at the Harvard Forest Flux Tower site during the 6 year (2001–2006) calibration and the 4 year (2007–2010) validation periods. Observed NECB is minus Net Ecosystem Exchange (NEE). Simulations noted with SCE (marked in green) and FME (marked in red) corresponded to the optimal parameter sets derived by the two methods, respectively. Sensitivity ranges of monthly GPP, NECB, ET, and LAI based on parameter distribution as generated with the Markov Chain Monte Carlo (MCMC) application. The light gray shade by Min-Max represents the minimum and maximum model response at each time step, whereas the dark gray shade by Mean 6 SD refers to the mean model response plus/minus one standard deviation. The vertical dash line marked the first month (January) of each year.
PPDF1 by both SCE and FME was just very close to its lower bound (28 C), but we did not plan to use a further lower value considering the physical meaning is more important than a better parameter value even if it may give better model performance. Figure 2 shows the monthly time series of observed and simulated four variables (i.e., GPP, NECB, ET, and LAI) during both the calibration and validation periods. This visual
WU ET AL.
C 2014. The Authors. V
412
Journal of Advances in Modeling Earth Systems
10.1002/2013MS000298
Table 3. Evaluation of Model Performance During the 6 Year (2001–2006) Calibration and 4 Year (2007–2010) Validation Periods Variable
Method
GPP
SCE FME
NECB
SCE FME
ET
SCE FME
LAI
SCE FME
Period
PB (%)a
NSEb
r2c
RSRd
Calibration Validation Calibration Validation Calibration Validation Calibration Validation Calibration Validation Calibration Validation Calibration Validation Calibration Validation
24.2 211.1 213.6 220.2 65.0 20.2 50.3 6.2 27.4 212.8 27.5 212.9 2.2 26.4 3.5 25.2
0.93 0.87 0.89 0.82 0.85 0.88 0.87 0.87 0.85 0.79 0.85 0.79 0.75 0.54 0.74 0.54
0.94 0.89 0.94 0.89 0.89 0.89 0.89 0.88 0.86 0.82 0.86 0.82 0.81 0.68 0.79 0.64
0.26 0.35 0.33 0.42 0.38 0.34 0.36 0.35 0.38 0.45 0.38 0.46 0.48 0.67 0.49 0.67
a
PB: percent bias (%). NSE: Nash-Sutcliffe Efficiency [Nash and Sutcliffe, 1970]. c 2 r : squared correlation coefficient [Krause et al., 2005]. d RSR: Root-Mean-Square Error (RMSE)—observation Standard deviation Ratio, which is a ratio of RMSE to observation standard deviation [Moriasi et al., 2007; Singh et al., 2005]. b
inspection can provide subjective estimates of the ‘‘closeness’’ of simulations and observations. From the figure, the simulations of the four variables with the calibrated parameter values by SCE and FME were found to be quite close and matched well with their corresponding observations. In addition to the visual comparison, Table 3 lists the statistical measures for the four variables during both calibration and validation periods in this study. For GPP simulations using the parameter values derived by SCE, NSE can reach 0.93 and 0.89 for calibration and validation, respectively, with |PB|
