WATER RESOURCES RESEARCH, VOL. 49, 6090–6095, doi:10.1002/wrcr.20437, 2013
Improving parameter priors for data-scarce estimation problems Susana Almeida,1 Nataliya Bulygina,1 Neil McIntyre,1,2 Thorsten Wagener,3 and Wouter Buytaert1 Received 25 January 2013; revised 10 June 2013; accepted 18 July 2013; published 4 September 2013.
[1] Runoff prediction in ungauged catchments is a recurrent problem in hydrology. Conceptual models are usually calibrated by defining a feasible parameter range and then conditioning parameter sets on observed system responses, e.g., streamflow. In ungauged catchments, several studies condition models on regionalized response signatures, such as runoff ratio or base flow index, using a Bayesian procedure. In this technical note, the Model Parameter Estimation Experiment (MOPEX) data set is used to explore the impact on model performance of assumptions made about the prior distribution. In particular, the common assumption of uniform prior on parameters is shown to be unsuitable. This is because the uniform prior on parameters maps onto skewed response signature priors that can counteract the valuable information gained from the regionalization. To address this issue, we test a methodological development based on an initial transformation of the uniform prior on parameters into a prior that maps to a uniform response signature distribution. We demonstrate that this method contributes to improved estimation of the response signatures. Citation: Almeida, S., N. Bulygina, N. McIntyre, T. Wagener, and W. Buytaert (2013), Improving parameter priors for data-scarce estimation problems, Water Resour. Res., 49, 6090–6095, doi:10.1002/wrcr.20437.
1.
Introduction
[2] In hydrological modeling, prior knowledge about the physical system is often represented by defining prior ranges of feasible parameter values [McIntyre et al., 2005; B ardossy, 2007; Kollat et al., 2012; Samuel et al., 2012]. Estimation often proceeds using Bayes’ equation (while there are less formal estimation methods, this note focuses primarily on a formal use of Bayes’). For gauged catchments, measured hydrological states and fluxes, usually flows, together with assumptions about error properties are used to calculate the likelihoods. For ungauged catchments, regionalized data can instead be used to feed the likelihood function. Signatures of catchment responses, which reflect hydrological response characteristics of a particular catchment (e.g., mean annual discharge, base flow index and runoff ratio), are commonly used forms of regionalized data. Estimates of such signatures are calculated for numerous gauged catchments and regressed against a set of catchment descriptors; thus, the signature distributions for an ungauged location, for which the same descriptors are available, can be estimated and used to calculate the likelihood [Bulygina et al., 2009, 2012]. Developing the use of regionalized response signatures for hydrological model 1
Department of Civil and Environmental Engineering, Imperial College London, London, UK. 2 Centre for Water in the Minerals Industry, Sustainable Minerals Institute, University of Queensland, Brisbane, Queensland, Australia. 3 Department of Civil Engineering, University of Bristol, Bristol, UK. Corresponding author: S. Almeida, Department of Civil and Environmental Engineering, Imperial College London, London SW7 2AZ, UK. (
[email protected]) ©2013. American Geophysical Union. All Rights Reserved. 0043-1397/13/10.1002/wrcr.20437
estimation is recognized as an important scientific challenge because the vast majority of catchments are ungauged, and the method also has potential for prediction under environmental change [Buytaert and Beven, 2009; Wagener and Montanari, 2011]. [3] When using a Bayesian procedure, assumptions about the prior distribution can have a considerable influence on the estimation of model parameters, especially in cases where data to calculate the likelihood are few and/or have high variance. It is therefore important not to introduce unintended or unjustified information into the prior, and in this respect it is common to exercise caution and aim for a noninformative prior. Introducing unjustified information can result in a disinformative prior that can counteract the information gained from the regionalization. Beven and Westerberg [2011] highlight that disinformation (in the context of disinformative data that ideally should be isolated and rejected prior to model calibration) should be avoided by all means. In a Bayesian context, a parametric prior is often expressed as a uniform distribution of parameters [Winsemius et al., 2009; Bulygina et al., 2011], where all parameter values fall within plausible bounds and different realizations of model parameter sets are considered equally likely a priori. However, as will be shown in this technical note, there are classes of problems where the prior is more usefully defined in terms of a uniform distribution of system behavior and where uniform distributions of model parameters should therefore not be used as the universal definition of prior lack of knowledge [Carlin and Louis, 2009]. [4] The problem of the prior distribution choice when little or no prior information is available has received much attention in the Bayesian statistical literature [Box and Tiao, 1992; Robert, 2007]. In many applications, uniform prior on parameters has been applied in an attempt to
6090
ALMEIDA ET AL.: TECHNICAL NOTE
reflect equiprobability. However, a uniform prior does not always reflect prior ignorance, as it is not invariant under reparameterization. In the Bayesian literature, several methods have been suggested to formulate invariant noninformative priors, such as the Jeffreys prior [Jeffreys, 1946, 1961] and the reference priors [Bernardo, 1979; Berger and Bernardo, 1989] (for an overview of methods for constructing noninformative priors, see Kass and Wasserman [1996]). However, these methods may require determination of complex derivatives, and therefore are often not straightforward to apply in a hydrological modeling context, where we are dealing with multidimensional and not necessarily continuous problems. Alternatively, Box and Tiao [1992] introduced what they call data-translated likelihood, which gives a very intuitive idea of what makes a uniform prior noninformative. For the problem tackled in this technical note, model parameters are conditioned on regionalized data and likelihood is expressed in terms of modeled response signatures. We argue here that we should express our initial lack of knowledge in the same space that we are getting information on, i.e., the response signature space, thus avoiding introducing unjustified information from other sources (e.g., the subjective choice of model structure or prior parameter distribution). Due to the difficulties associated with approaches such as Jeffreys and reference priors in this particular context, we suggest here to use a uniform prior on response signature space to avoid disinformation. This contrasts with the usual practice in hydrological modeling to assume a uniform prior on parameters, especially when using conceptual-type models where the parameters have limited physical interpretation and thus limited prior knowledge. It is generally not possible to sample directly from a uniform signature distribution. In this technical note, we therefore propose a method for transforming an initial uniform prior on parameters to a distribution that maps to a uniform prior in terms of response signatures. A set of 84 catchments in the eastern USA, taken from the Model Parameter Estimation Experiment (MOPEX) database [Duan et al., 2006], for which a variety of regional response signature models and likelihood functions have previously been determined [Almeida et al., 2012], are used to test the method. The potential value of the method for a wider range of hydrological applications is subsequently discussed.
2.
Method
2.1. Bayes’ Method for Model Conditioning [5] We use a Bayesian procedure to condition a model on data about expected values of response signatures and their uncertainty. Although the procedure is relevant for other types of model and sources of data, the particular problem examined here is that of conditioning the parameters of a conceptual hydrological model on regionalized response signatures. Bayes’ law here is expressed as pðHjs ; I; M Þ ¼
LðsðHÞjs ; I; M Þ pðHjI; M Þ pðs jI; M Þ
ð1Þ
where, for one catchment, s represents the regionalized response signature data upon which the prior model is con-
ditioned; pðHjI; M Þ is the prior distribution of parameters H for model structure M and the catchment’s set of time series inputs I; LðsðHÞjs ; I; M Þ is the likelihood function of the modeled response signature sðHÞ given s , I and M; pðs jI; M Þ is the marginal density of s ; and pðHjs ; I; M Þ is the posterior distribution of H given s , I, and M. For the purpose of this note, M is selected in advance and considered to be fixed, as is I for any one catchment, and so both these terms will be dropped from equation (1) (the implications of dropping M and I are discussed later) : pðHjsÞ ¼
LðsðHÞjsÞ pðHÞ pðsÞ
ð2Þ
[6] The model used in the case study is the probability distributed moisture (PDM) model [Moore, 2007] together with two parallel linear routing stores and a simple snow model [Hock, 2003]. [7] To apply Bayes’ law, it is necessary to specify the likelihood function and the prior. The likelihood function in the case study is derived explicitly by analyzing the distribution of the errors from the original regionalization model of Almeida et al. [2012]. For more details on how the likelihood was derived, the reader is referred to Almeida et al. [2012]. This technical report focuses on parameter priors. 2.2. A Uniform Prior on Parameters [8] The usual assertion in conceptual hydrological modeling—that there is sufficiently little prior information about the model parameters that their priors should be uniform and independent of one another—is used here as a starting point. The joint uniform prior on parameters is denoted by pup ðHÞ. N parameter sets are randomly drawn from this prior (we chose N ¼ 1000 as our results showed limited sensitivity to larger values of N). Using the preselected model structure M, and relevant time series inputs I, N hydrographs are generated for a catchment of interest. For each of these hydrographs, the corresponding response signature is computed and the likelihood value calculated. The parameter and signature posteriors, pðHjsÞ and pðsðHÞjsÞ, are thus approximated from the N samples, and pðsðHÞjsÞ can then be assessed in terms of how successfully it explains the distribution of observed response signatures (using the metric described in section 2.4). This provides a benchmark to assess the performance of an alternative prior. 2.3. A Parameter Prior That Maps Onto a Uniform Response Signature Distribution [9] The premise of this technical note is that when a model is conditioned on information coming from regionalized response signatures, a uniform prior on parameters pup ðHÞ is disinformative and instead equation (2) should be applied assuming a prior that maps (by running it through the model M) onto a uniform prior on response signature, pus ðHÞ. Since it is generally not possible to sample directly from pus ð:Þ, the distribution pus ð:Þ is approximated using the N parameter samples from pup ð:Þ and corresponding importance weights [Doucet et al., 2000]. The importance weights are calculated using equation (3)
6091
ALMEIDA ET AL.: TECHNICAL NOTE
Figure 1. A uniform parameter mapping onto the response signature distribution, sðHÞ, the likelihood, LðsðHÞjsÞ, and best estimate value, s : example of the Nezinscot River at Turner Center, Maine.
pus ðHÞ ¼ C
pup ðHÞ g ½sðHÞ
ð3Þ
where g ð:Þ is a response signature probability distribution derived by mapping (by running through the model M) a uniform parameters distribution onto the response signature space, and C is a normalizing constant to guarantee that the integral of pus ðHÞ with respect to H is one. Division by the response signature distribution g ð:Þ downweights frequent signature values and increases the weight of less frequent signature values. Since the mapped response signature distribution (in most cases) cannot be derived analytically, it is approximated by mapping N parameter samples from pup ð:Þ onto the signature space via model M. Distribution of
the resulting signature draws can then be approximated using a Kernel density approximation [Silverman, 1986], histogram, or, as implemented here, using a mixture of Gaussian distributions parsimoniously parameterized by means of a Dirichlet process model [Muller et al., 1996]. The upper and lower bounds for pus ð:Þ are dependent on the model structure choice, model parameter space definition, as well as model inputs. [10] With a suitable pus ð:Þ now estimated, equation (2) can be applied to estimate the parameter and response signature posteriors, pðHjsÞ and pðsðHÞjsÞ. The latter’s performance in explaining the observed distribution of response signatures is compared with the benchmark performance.
6092
ALMEIDA ET AL.: TECHNICAL NOTE
Figure 2. QQ plots comparing the available sample zi with theoretical quantiles of U(0,1) for each of the five signatures. 2.4. Performance Assessment [11] There are numerous methods that might be used for comparing the observed response signatures with the two alternative modeled distributions. The graphical QQ plot method suggested by Laio and Tamea [2007] is adapted here. The response signature predictions are considered to be consistent with the observed response signature ^s i , where i refers to a specific catchment, if the modeled response signature cumulative distribution function Pi evaluated at the observed response signature, zi ¼ Pi ð^s i Þ, comes from a uniform distribution U(0,1) [Laio and Tamea, 2007]. For the purpose of testing whether a uniform distribution is achieved, Laio and
Tamea [2007] suggest a graphical method based on a QQ plot. A QQ plot compares the available sample of zi with the theoretical quantiles of U(0,1). If the two probability distributions are similar, the plot will be close to the 1 :1 line. Here, we use a jack-knife approach that provides 84 samples of z corresponding to the 84 test catchments : one catchment at a time is removed as the test ‘‘ungauged’’ catchment and the remaining 83 catchments are used to support the regionalization and thus to determine the test catchment’s posterior and hence z value. This process is repeated for all catchments and the performance integrated across the 84 catchments is assessed using the QQ plot.
6093
ALMEIDA ET AL.: TECHNICAL NOTE
[12] The performance is assessed using five different response signatures—runoff ratio (RR), base flow index (BFI), streamflow elasticity (SE), slope of flow duration curve (SFDC), and high pulse count (HPC). For a description of these signatures, see Yadav et al. [2007] and Sawicz et al. [2011]. Although the five response signatures could be treated together to define a joint prior and posterior distribution, they are used individually here to evaluate the proposed method for different responses.
3.
Results
[13] Using a 439 km2 subcatchment of the Nezinscot River, Maine, as an example, Figure 1 shows that uniform prior on parameters maps onto significantly skewed response signature priors (i.e., prior to introducing information from the regionalized signature estimates). Figure 1 also shows, for the same catchment, an expected value and likelihood function for each response signature, illustrating that, in particular for RR and SE, there is information in the prior that is inconsistent with the information coming from the regionalization. Similar skewness and directions of inconsistencies were prevalent over the 84 catchments. [14] Figure 2 shows QQ plots obtained using the uniform prior on parameters (shown using subscript up), demonstrating that in general across the 84 catchments there is no consistent agreement between modeled and observed response signature values. This supports the hypothesis that the uniform prior on parameters is disinformative, unintentionally giving undue weight to some regions of the output response signature space. [15] Figure 2 also shows QQ plots obtained when the prior that maps onto a uniform response signature distribution is employed (shown using subscript us). This result suggests that better overall performance can be achieved by using this type of prior. For BFI and HPC, similar results were obtained with either prior, because the uniform prior on parameters tends to map onto more uniform prior on signatures (Figure 1).
4.
Assumptions and Applicability
[16] It may be argued that the preselection of the model structure implies that prior information about the hydrological system is being used and therefore it is not appropriate to transform the prior distributions so that they are noninformative with respect to the particular signatures being simulated. However, in this case study, as in many other regionalization studies, the model structure was chosen from a position of limited knowledge about how well it can replicate responses over a large number of catchments. The method accepts that more prior weight should be given to parameters that compensate for unjustified preconceptions of the model structure about flow responses. [17] It may also be argued that the meteorological data should provide some prior knowledge about the nature of the response signatures. In this method, the influence of the meteorology is included in the signature regionalization and thus included in the likelihood function. On that basis, it was assumed valid to omit it from the prior. [18] A further limitation of the method proposed here is that the prior signature distributions are bounded, which
introduces some information in an unbounded signature case (i.e., SE, SFDC, and HPC). However, a very similar problem arises when choosing parameter bounds for a uniform prior on parameters. Both problems are alleviated by choosing ‘‘wide enough’’ bounds, allowing for signature bounds to be physically meaningful and grounded in the literature [Sankarasubramanian and Vogel, 2003]. [19] This note relates to problems where the hydrological model does not lend itself to the specification of parameters prior to the integration of information via Bayes’ theorem. This includes a range of conceptual hydrological models, where it is the common practice (but not necessarily good practice, as shown here) to specify a prior as being uniform on parameters. There are other situations where it may be considered desirable to use an informative parameter prior because it accounts for information considered to be important and valid due to long-term experience with a particular model [Kapangaziwiri et al., 2012]. [20] This note also relates to catchments with sparse information, for example only a few, uncertain observations or regionalized response signatures to condition the model upon. Where the likelihood function encompasses many more independent items of information, as is sometimes considered to be the case when conditioning upon time series of observed flow [Sorooshian and Dracup, 1980], the likelihood function may overwhelm the effect of the prior irrespective of its distribution. However, in most hydrological applications, even where time series of flow exist, often the items of information feeding into the likelihood function can neither be considered independent nor relatively precise, increasing the importance of the prior.
5.
Conclusions
[21] Ensuring a suitably noninformative prior distribution is often neglected when applying Bayes’ theorem. We show in this note the potential importance of avoiding disinformative priors using a rainfall-runoff modeling case study where data were scarce and uncertain, coming only from a regionalization exercise. An initial transformation of the uniform prior on parameters into a prior that maps to a uniform response signature distribution contributed to improved estimation of the catchment response signatures. It is speculated that such a method may improve the success of Bayesian conditioning for a range of data-scarce model applications. [22] Acknowledgments. The authors would like to acknowledge the support of Fundac¸ ~ ao para a Ci^encia e a Tecnologia (FCT), Portugal, sponsor of the PhD program of S.A. at Imperial College London, under the grant SFRH/BD/65522/2009. The authors would also like to acknowledge the hydrology group at Pennsylvania State University, in particular Keith Sawicz, for advice and support relating to the data used in this study. Time series data used in this study was provided by the MOPEX project. The authors thank David Huard and two other anonymous reviewers for their useful comments.
References Almeida, S. L., N. Bulygina, N. McIntyre, T. Wagener, and W. Buytaert (2012), Predicting flows in ungauged catchments using correlated information sources, paper presented Eleventh National Hydrology Symposium: Hydrology for a Changing World, Br. Hydrol. Soc., Dundee, U. K. [Available at http://www.hydrology.org.uk/publications.php.].
6094
ALMEIDA ET AL.: TECHNICAL NOTE Bardossy, A. (2007), Calibration of hydrological model parameters for ungauged catchments, Hydrol. Earth Syst. Sci., 11(2), 703–710, doi:10.5194/hess-11-703-2007. Berger, J. O., and J. M. Bernardo (1989), Estimating a product of means— Bayesian-analysis with reference priors, J. Am. Stat. Assoc., 84(405), 200–207, doi:10.2307/2289864. Bernardo, J. M. (1979), Reference posterior distributions for bayesian-inference, J. R. Stat. Soc. B Met., 41(2), 113–147. Beven, K., and I. Westerberg (2011), On red herrings and real herrings: Disinformation and information in hydrological inference, Hydrol. Processes, 25(10), 1676–1680, doi:10.1002/Hyp.7963. Box, G., and G. Tiao (1992), Bayesian Inference in Statistical Analysis, John Wiley, New York. Bulygina, N., N. McIntyre, and H. Wheater (2009), Conditioning rainfallrunoff model parameters for ungauged catchments and land management impacts analysis, Hydrol. Earth Syst. Sci., 13(6), 893–904, doi:10.5194/ hess-13-893-2009. Bulygina, N., N. McIntyre, and H. Wheater (2011), Bayesian conditioning of a rainfall-runoff model for predicting flows in ungauged catchments and under land use changes, Water Resour. Res., 47, W02503, doi:10.1029/2010WR009240. Bulygina, N., C. Ballard, N. McIntyre, G. O’Donnell, and H. Wheater (2012), Integrating different types of information into hydrological model parameter estimation: Application to ungauged catchments and land use scenario analysis, Water Resour. Res., 48, W06519, doi:10.1029/2011WR011207. Buytaert, W., and K. Beven (2009), Regionalization as a learning process, Water Resour. Res., 45, W11419, doi:10.1029/2008WR007359. Carlin, B. P., and T. A. Louis (2009), Bayesian Methods for Data Analysis, 3rd ed., Chapman and Hall, Boca Raton, Fla. Doucet, A., S. Godsill, and C. Andrieu (2000), On sequential Monte Carlo sampling methods for Bayesian filtering, Stat. Comput., 10(3), 197–208, doi:10.1023/a:1008935410038. Duan, Q., et al. (2006), Model Parameter Estimation Experiment (MOPEX): An overview of science strategy and major results from the second and third workshops, J. Hydrol., 320(1–2), 3–17, doi:10.1016/ j.jhydrol.2005.07.031. Hock, R. (2003), Temperature index melt modelling in mountain areas, J. Hydrol., 282(1–4), 104–115, doi:10.1016/S0022-1694(03)00257-9. Jeffreys, H. (1946), An invariant form for the prior probability in estimation problems, Proc. R. Soc. London, Ser. A, 186(1007), 453–461, doi:10.2307/97883. Jeffreys, H. (1961), Theory of Probability, 3rd ed., Oxford Univ. Press, Oxford, U. K. Kapangaziwiri, E., D. A. Hughes, and T. Wagener (2012), Incorporating uncertainty in hydrological predictions for gauged and ungauged basins in southern Africa, Hydrol. Sci. J., 57(5), 1000–1019, doi:10.1080/ 02626667.2012.690881.
Kass, R. E., and L. Wasserman (1996), The selection of prior distributions by formal rules, J. Am. Stat. Assoc., 91(435), 1343–1370, doi:10.2307/ 2291752. Kollat, J. B., P. M. Reed, and T. Wagener (2012), When are multiobjective calibration trade-offs in hydrologic models meaningful?, Water Resour. Res., 48, W03520, doi:10.1029/2011WR011534. Laio, F., and S. Tamea (2007), Verification tools for probabilistic forecasts of continuous hydrological variables, Hydrol. Earth Syst. Sci., 11(4), 1267–1277, doi:10.5194/hess-11-1267-2007. McIntyre, N., H. Lee, H. Wheater, A. Young, and T. Wagener (2005), Ensemble predictions of runoff in ungauged catchments, Water Resour. Res., 41, W12434, doi:10.1029/2005WR004289. Moore, R. J. (2007), The PDM rainfall-runoff model, Hydrol. Earth Syst. Sci., 11(1), 483–499, doi:10.5194/hess-11-483-2007. Muller, P., A. Erkanli, and M. West (1996), Bayesian curve fitting using multivariate normal mixtures, Biometrika, 83(1), 67–79, doi:10.1093/ biomet/83.1.67. Robert, C. (2007), The Bayesian Choice : From Decision-Theoretic Foundations to Computational Implementation, 2nd ed., Springer, New York. Samuel, J., P. Coulibaly, and R. A. Metcalfe (2012), Identification of rainfall-runoff model for improved baseflow estimation in ungauged basins, Hydrol. Processes, 26(3), 356–366, doi:10.1002/Hyp.8133. Sankarasubramanian, A., and R. M. Vogel (2003), Hydroclimatology of the continental United States, Geophys. Res. Lett., 30(7), 1363, doi:10.1029/ 2002GL015937. Sawicz, K., T. Wagener, M. Sivapalan, P. A. Troch, and G. Carrillo (2011), Catchment classification: Empirical analysis of hydrologic similarity based on catchment function in the eastern USA, Hydrol. Earth Syst. Sci., 15(9), 2895–2911, doi:10.5194/hess-15-2895-2011. Silverman, B. W. (1986), Density Estimation for Statistics and Data Analysis, Chapman and Hall, London. Sorooshian, S., and J. A. Dracup (1980), Stochastic parameter-estimation procedures for hydrologic rainfall-runoff models—correlated and heteroscedastic error cases, Water Resour. Res., 16(2), 430–442, doi:10.1029/ WR016i002p00430. Wagener, T., and A. Montanari (2011), Convergence of approaches toward reducing uncertainty in predictions in ungauged basins, Water Resour. Res., 47, W06301, doi:10.1029/2010WR009469. Winsemius, H. C., B. Schaefli, A. Montanari, and H. H. G. Savenije (2009), On the calibration of hydrological models in ungauged basins: A framework for integrating hard and soft hydrological information, Water Resour. Res., 45, W12422, doi:10.1029/2009WR007706. Yadav, M., T. Wagener, and H. Gupta (2007), Regionalization of constraints on expected watershed response behavior for improved predictions in ungauged basins, Adv. Water Resour., 30(8), 1756–1774, doi:10.1016/j.advwatres.2007.01.005.
6095
