JANUARY 2014
VAN LIER-WALQUI ET AL.
401
Linearization of Microphysical Parameterization Uncertainty Using Multiplicative Process Perturbation Parameters MARCUS VAN LIER-WALQUI Rosenstiel School of Marine and Atmospheric Science, University of Miami, Miami, Florida
TOMISLAVA VUKICEVIC Hurricane Research Division, NOAA/AOML, Miami, Florida
DEREK J. POSSELT University of Michigan, Ann Arbor, Ann Arbor, Michigan (Manuscript received 22 February 2013, in final form 5 June 2013) ABSTRACT Recent studies have shown the importance of accounting for model physics uncertainty within probabilistic forecasts. Attempts have been made at quantifying this uncertainty in terms of microphysical parameters such as fall speed coefficients, moments of hydrometeor particle size distributions, and hydrometeor densities. It has been found that uncertainty in terms of these ‘‘traditional’’ microphysical parameters is highly nonGaussian, calling into question the possibility of estimating and propagating this error using Gaussian statistical techniques such as ensemble Kalman methods. Here, a new choice of uncertain control variables is proposed that instead considers uncertainty in individual modeled microphysical processes. These ‘‘process parameters’’ are multiplicative perturbations on contributions of individual modeled microphysical processes to hydrometeor time tendency. The new process parameters provide a natural and appealing choice for the quantification of aleatory microphysical parameterization uncertainty. Results of a nonlinear Monte Carlo parameter estimation experiment for these new process parameters are presented and compared with the results using traditional microphysical parameters as uncertain control variables. Both experiments occur within the context of an idealized one-dimensional simulation of moist convection, under the observational constraint of simulated radar reflectivity. Results indicate that the new process parameters have a more Gaussian character compared with traditional microphysical parameters, likely due to a more linear control on observable model evolution. In addition, posterior forecast distributions using the new control variables (process parameters) are shown to have less bias and variance. These results strongly recommend the use of the new process parameters for an ensemble Kalman-based estimation of microphysical parameterization uncertainty.
1. Introduction Parameterizations are employed in numerical weather prediction to model unresolved subgrid-scale processes such as microphysical processes controlling clouds and precipitation. The parameters of these schemes are often chosen empirically (i.e., based on limited observations; Kessler 1969); in addition, their optimal (most likely) value can vary within the scale of the convective system
Corresponding author address: Marcus van Lier-Walqui, RSMAS, University of Miami, 4600 Rickenbacker Cswy., Miami, FL 33130. E-mail:
[email protected] DOI: 10.1175/MWR-D-13-00076.1 Ó 2014 American Meteorological Society
(Morrison et al. 2009). Indeed, it should be emphasized that such parameters do not have a correct value, only a most likely value, as all values would result in some degree of forecast error. Thus, these parameters may have both epistemic uncertainty (their optimal value is unknown) as well as aleatory uncertainty (they have an inherent and irreducible uncertainty or stochasticity). Meanwhile, it has been suggested that probabilistic forecast ensembles that do not include model physics uncertainty may not have sufficient dispersion (Stensrud et al. 2000). These concerns motivate the study of model physics uncertainty quantification. The goal is an improved ensemble forecast— one that more accurately predicts forecast uncertainty.
402
MONTHLY WEATHER REVIEW
Recent studies have quantified uncertainty in terms of ‘‘traditional’’ microphysical parameters such as hydrometeor densities and fall speed parameters as well as parameters of the hydrometeor particle size distribution. Such uncertainty has been found to be highly nonGaussian, whether constrained by column-integral model output variables (Posselt and Vukicevic 2010, hereafter PV10) or vertically resolved radar reflectivity (van LierWalqui et al. 2012, hereafter VVP12). In addition, Vukicevic and Posselt (2008) and Posselt and Bishop (2012) indicate that inversion methods that assume Gaussian statistics in the posterior distribution will inherently misrepresent non-Gaussian parameter uncertainty, resulting in the possibility of biased estimates of parameter values and biased forecast ensembles. These facts stand in the way of a full account of model physics uncertainty involving 3D models and real data, because a computationally tractable uncertainty estimation would have to necessarily involve methods that assume Gaussian error statistics such as ensemble Kalman filters and derivatives. One option for addressing this issue is to instead implement a nonlinear data assimilation framework (van Leeuwen 2010; Hodyss 2011; Sondergaard and Lermusiaux 2013). Another method is to search for a new, different choice of uncertain control variables that would support the Gaussian error statistics. In this study, the latter approach is chosen. Results from VVP12 show that, while uncertainty in the traditional microphysical parameters is highly non-Gaussian, the posterior distributions of microphysical process tendency terms in the parameterization scheme tend to display reduced skewness and multimodality (see, e.g., Figs. 9 and 10 of VVP12). One explanation is that the traditional parameters in microphysics schemes control numerous simulated processes, resulting in a relationship with observable quantities, which is in some cases nonmonotonically nonlinear and strongly dependent on storm morphology. By contrast, individual modeled processes, and their associated hydrometeor tendency terms, directly control hydrometeor mixing ratio tendency. Because observations such as radar reflectivity are sensitive to the concentration and type of hydrometeors, nonlinearity in the relationship between perturbations of these processes and the response of an observable quantity can be expected to be largely attributable to nonlinearity in the observational forward operator, such as the relationship between radar reflectivity and hydrometeor concentrations. A quantification of microphysical parameterization model error in terms of process uncertainty rather than parameter uncertainty can also be seen as a natural choice in that it agrees with a subjective assessment of how models misrepresent true microphysics. Estimating
VOLUME 142
uncertainty in terms of the traditional microphysical parameters implicitly assumes no uncertainty in the parameterized assumptions that employ these coefficients. By contrast, uncertainty in processes may be the result of uncertainty in those parameters or uncertainty in the functional form of the assumptions that employ them, thus better capturing aleatory parameterization uncertainty.
2. Approach New parameters are proposed that multiplicatively perturb the contribution by modeled microphysical processes to hydrometeor tendency. It is hypothesized that these new ‘‘process parameters’’ may provide a basis for representing microphysical model error, which is appropriate for use with Gaussian data assimilation methods such as ensemble Kalman methods. The essential questions of this research are as follows: 1) Is uncertainty in terms of process parameters more Gaussian than for traditional microphysical parameters? 2) Does uncertainty estimated in terms of the new process parameters result in improved posterior forecasts? The first question is answered by performing a Monte Carlo inversion on the process parameters of interest, given an observational constraint. The PDF generated by a Monte Carlo technique is a direct result of the nonlinear relationship between the control variables of interest and the observable quantity, constrained by an uncertain observation. In general, the Monte Carlo approach allows for non-Gaussian or multimodal solutions. The posterior distribution can be tested for Gaussianity by analyzing the degree to which it matches a Gaussian distribution. One metric, presented in Mardia (1970) is a multivariate analog of the univariate measures of skewness and kurtosis. Lack of multivariate skewness and kurtosis are necessary, but not sufficient, conditions for the Gaussianity of a multivariate distribution. Another method for testing Gaussianity is to analyze the Mahalanobis distance of samples of the posterior distribution. The Mahalanobis distance is measure of distance of points in a distribution from the distribution mean, scaled by the covariance (Thompson 1990). Mahalanobis distance allows for succinct analysis of a distribution’s deviations from a multivariate Gaussian, deviations that may not be obvious from inspection of marginals of that distribution. The second question is answered by considering the posterior distributions of various model forecast variables. Specifically, the distributions of the following quantities are analyzed: (i) the contribution of modeled microphysical processes to hydrometeor time tendency, (ii) column-integral model outputs that provide the
JANUARY 2014
VAN LIER-WALQUI ET AL.
observational constraint in PV10, and (iii) simulated radar reflectivity that provides the observational constraint in VVP12 and the current experiment. These posterior distributions are analyzed for sensitivity to perturbation of the model control variables, as well as constraint and bias of the posterior marginal distributions relative to truth. Shannon information gain is calculated for the posterior distribution of the columnintegral model outputs, for both inverse experiments using traditional microphysical parameters (VVP12) as well as the new process parameters of the current experiments. The Shannon information gain is a metric used, for example, in the atmospheric sciences for quantifying the information content of observational retrievals (Shannon 1948; Posselt et al. 2008; Vukicevic et al. 2010; Coddington et al. 2012). Information gain measures the decrease in informational entropy, or disorder, from the prior distribution to the posterior distribution of some variable. Maximum entropy is obtained via a uniform probability distribution, where no state is more probable than any other, whereas minimum entropy is defined as a Dirac delta function, where only one value of the variable(s) has nonzero probability. As the constraint provided by observations sharpens a posterior distribution relative to the prior distribution, information is gained on the variables over which the PDF is defined. Here, it is posited that in order for uncertain variables to indicate control of the model solution, they must gain information from the observational constraint. Information gain of the posterior is calculated relative to a common uninformative distribution—a bounded uniform distribution spanning the common range of values of each dataset. Shannon information gain is calculated for the column-integral model output variables because these variables are used neither as constraining observations nor model control variables in both the current study and VVP12.
3. Experiment a. Inversion method The forward model used in this experiment consists of an atmospheric one-dimensional cloud-simulating model coupled with a radar reflectivity simulator. The radar observations and associated observational uncertainty are described in VVP12 and the one-dimensional column atmospheric model is described in detail in PV10. Briefly, the atmospheric model is based on the Goddard Cumulus Ensemble model (Tao and Simpson 1993; Tao et al. 2003), modified to function in a single vertical dimension, which is forced by time-varying vertical water vapor and vertical velocity tendency profiles. The forcing functions,
403
described in PV10, are chosen so as to produce microphysical behavior similar to that observed within a midlatitude squall line with trailing stratiform. The temporal evolution of the storm is meant to replicate the conditions of a column of air that is drawn along the cross-front dimension of a two- or three-dimensional squall line. The model uses the Tao et al. (2003) single moment microphysics parameterization scheme. The observation variables are simulated 3-GHz radar reflectivity, calculated using the Quickbeam radar simulator (Haynes et al. 2007), which assumes Mie theory to calculate reflectivity for user-specified radar frequency and atmospheric state. The user may also specify the parameterization used so that the radar simulator is consistent with microphysical parameterization assumptions in the model. Quickbeam, in the configuration used for the current research, does not simulate beam bending, beam broadening or radar brightband effects. Observational error is assumed to be vertically correlated, time-dependent, with a distribution defined by the ensemble of model runs generated in PV10. This assumption amounts to considering the parameter uncertainty in PV10 to be a ‘‘climatology’’ from which to draw observational error statistics. The radar reflectivity error covariance matrices used are shown in Fig. 3 of VVP12. The inverse parameter estimation experiment is performed for the one-dimensional idealized squall-line simulation using the simulated radar reflectivity from a truth simulation as observational constraint. The result of the inversion is a posterior PDF in a multidimensional control variable space. This posterior PDF represents P(x j y, M), the probability of some choice of parameters x given observations vector y and model assumptions M. The control parameters of the model, for which uncertainty is estimated, have been changed from the traditional microphysical parameters of PV10 and VVP12 to multiplicative factors on individual tendency contributions within the microphysical parameterization code. Over 40 distinct simulated processes exist within the Tao et al. (2003) microphysical parameterization scheme. For computational tractability, 12 of the most active processes, listed in Table 1, were chosen from the full set based on their activity in a truth simulation. This truth simulation is identical to that of PV10 and VVP12 (i.e., with no perturbation on the parameters from a value defined here as the true value). For the new multiplicative process parameters, truth corresponds to a value of 1. In the inversion, perturbation of the parameters is controlled by a Monte Carlo sampling algorithm in order to efficiently estimate the relative probability distribution within the 12-dimensional space of these control parameters.
404
MONTHLY WEATHER REVIEW
VOLUME 142
TABLE 1. Microphysical process descriptions, abbreviation, source, and product hydrometeors of the process. Process description
Abbreviation
Source
Product
Evaporation of rain (negative) Melting of snow Melting of graupel Cloud ice accretion of rain Graupel accretion of rain Rain accretion of cloud water Rain accretion of snow Deposition on snow Bergeron process (deposition/riming) Snow accretion of rain Graupel accretion of cloud water Snow accretion of cloud water
ERN PSMLT PGMLT PIACR DGACR PRACW QRACS PSDEP PSFW PSACR DGACW PSACW
Rain Snow Graupel Cloud ice, rain Graupel, rain Rain, cloud water Rain, snow Water vapor, snow Cloud water Rain, snow Graupel, cloud water Cloud water, snow
Water vapor Rain Rain Snow, graupel Graupel Rain Rain Snow Snow Snow, graupel Graupel Snow, graupel
b. Inversion technique The Monte Carlo algorithm has been modified from the Markov chain Monte Carlo (MCMC) Metropolis sampler used in PV10 and VVP12 to a delayed-rejection, adaptive Metropolis sampler (DRAM; Mira 2001; Haario et al. 2001, 2006). In addition, a simulated annealing presampler is implemented prior to the main Monte Carlo inversion in order to optimize the start position of the main sampler. The simulated annealing algorithm does not sample from the posterior distribution, and therefore, these samples are discarded. These modifications have been made to improve the efficiency and reliability of the Monte Carlo inversion relative to commonly used MCMC samplers, but do not affect the robustness of the resulting posterior PDF. The DRAM algorithm is described in detail elsewhere (Mira 2001; Haario et al. 2001, 2006); however, the basic characteristics will be summarized here. DRAM is the implementation of two techniques: delayed rejection sampling (Mira 2001) and adaptive Metropolis sampling (Haario et al. 2001). Delayed rejection seeks to improve sampler performance, particularly for non-Gaussian posterior PDFs, by considering a second proposal x02 in the case that the first x01 is rejected. In this case, the second proposal is chosen using a reduced proposal variance— the assumption being that the first proposal may have strayed well outside the main posterior probability mass, and that a more ‘‘conservative’’ proposal is warranted. This second proposal is then accepted as a step from the original point x with a probability given by 2 3 P(x01 ) 1 2 min 1, 6 P(x0 ) P(x02 ) 7 6 7 2 (1) a2 (x, x02 ) 5 min61, 7. 4 P(x) 5 P(x01 ) 12 P(x) Note that for very low values of likelihood, it may be preferable to use log-likelihood (the cost function value)
in this equation to avoid numerical underflow [i.e., exp(Fx 2 Fx0 ) rather than P(x0 )/P(x), where P(x) } exp(Fx)]. Adaptive Metropolis is a modification of the Metropolis–Hastings algorithm (Metropolis et al. 1953; Hastings 1970), where the proposal distribution is adapted based on the covariance of all previous samples. This addresses situations where the algorithm samples inefficiently due to a too small or large a proposal covariance (Haario et al. 2001). It should be noted that because the adaptive proposal retains memory of all previous samples, DRAM is not Markovian, and is thus not, strictly speaking, an MCMC method. As in PV10 and VVP12, the prior parameter distribution is chosen to be a bounded uniform, to reflect a lack of prior information, while optimizing the performance of the Monte Carlo algorithm. In that study, the bounds of the bounded uniform prior distribution were chosen to match clearly unphysical values of the parameters, or values at which the model became numerically unstable. In the current work, no such insight exists, a priori. The multiplicative parameters are clearly positive definite, but no specific upper bound can be set without additional information or making assumptions. The positive definiteness of perturbation by process parameters is partly a consequence of the manner in which process activity is calculated in the microphysical code. For example, negative perturbations of deposition processes are obviated by the fact that sublimation is calculated independently. A prior process parameter range of 0–3 is chosen for all parameters. The lower limit of 0 is equivalent to a complete ‘‘turning off’’ of a microphysical process, and is an obvious lower limit for process perturbation. A value of 1 for the process parameters corresponds to an unperturbed process, and is the default value for the truth simulation. The upper limit of 3 is a tripling of the standard process activity calculated in the code. It should be noted that process activity is still limited by
JANUARY 2014
VAN LIER-WALQUI ET AL.
hydrometeor concentration and other physical considerations within the microphysics code, and so actual process activity may lose sensitivity to process parameter perturbation when the processes are so limited. The value of 3 for the upper limit was chosen because results were found to be largely insensitive to an expansion of the range past this point.
c. Experiment diagnostics A variety of techniques are used to characterize the performance of the inversion. As in VVP12, joint distributions of control variables and microphysical process activity are shown to display the characteristics of the relationship between control variable perturbation and model microphysical behavior. Of particular interest is whether the choice of process parameters exerts a linear control on the system. As discussed in section 2, Gaussianity of the posterior PDF is tested using squared Mahalanobis distance (Thompson 1990) and the Mardia (1970) test of multivariate skewness and kurtosis. Mahalanobis distance measures the distance of samples in a distribution from the center (mean, x) of that distribution, as scaled by the distribution covariance C. The squared Mahalanobis distance of a sample x is given by Mi2 5 (xi 2 x)T C21 (xi 2 x) .
(2)
For samples from a Gaussian PDF, these values are distributed as a chi-square distribution, and deviations from Gaussianity can be determined by plotting against chi-square quantiles. The Mardia (1970) tests of multivariate skewness and kurtosis are calculated to gain further information about the non-Gaussian structure of the posterior PDF. In addition, these measures help to partition and quantify the source of non-Gaussianity observed in the Mahalanobis distance. To quantify the ability of posterior control variable distributions to produce an informative forecast, Shannon information gain on independent metrics of simulated storm evolution is calculated relative to minimum information over these variables. The independent metrics chosen are the column-integral quantities used in PV10 as observational constraint, namely, precipitation rate (PRATE), liquid water path (LWP), ice water path (IWP), outgoing longwave radiation (OLW), and outgoing shortwave radiation (OSW). These variables are commonly used when comparing model performance to observations, and also are neither directly constrained nor used as observational constraint in the current inverse experiment. Information gain is calculated for the posterior univariate distributions of these variables, relative to an uninformative distribution across a range shared by the current inversion as well as that of VVP12. The
405
posterior distributions are also inspected for bias, which is not quantified by the Shannon information gain.
4. Results and discussion Figure 1 shows the posterior PDF on the 12 process parameters, for a prior parameter range of 0–3. As in VVP12, the full PDF is shown as a series of 1D and 2D joint marginal distributions, which are the result of integrating across 11 and 10 parameter dimensions, respectively. In these plots, warm colors, peaking at white, represent regions of the parameter space with high probability, while cooler colors represent lower probability parameter values. The location of parameter values for the truth simulation (parameter value 5 1 for all parameters) is marked by red crosshairs. These results show that process parameters such graupel accretion of rain (DGACR), rain accretion of snow (QRACS), Bergeron deposition on snow (PSFW), and snow accretion of rain (PSACR) have nearly uniform posterior marginal distributions, indicating weak constraint by observations over the prior parameter range specified. Other process parameters such as rain accretion of cloud water (PRACW), vapor deposition on snow (PSDEP), and dry growth of graupel via accretion of cloud water (DGACW) show strong constraint by the observations, with the mode of the associated marginals collocated with the truth position of the process parameters, indicating an unbiased and unambiguous inversion of these parameters. Evaporation of rain (ERN), snowmelt (PSMLT), graupel melt (PGMLT), cloud ice accretion of rain (PIACR), and snow accretion of cloud water (PSACW) show intermediate degrees of observational constraint, with ERN displaying bimodality and PSACW with significant bias in the marginal estimate of the model. ERN, PSMLT, PGMLT, and PIACR show noticeable skewness in their univariate marginal distributions. The skewness is an expected result with multiplicative perturbation parameters, which are positive definite, and so therefore cannot be normally distributed and are perhaps better described by a positive definite distribution such as a lognormal distribution [see a discussion of Jeffreys quantities in Tarantola (2005)]. Of greater concern is the bimodality shown for the marginal of ERN, which indicates that it is not possible to obtain an unambiguous estimate of the maximum likelihood estimate for this parameter because of nonmonotonic nonlinearity in the relationship between parameter perturbation and observable quantities. Similarly, the strong bias shown in the marginal distribution of PSACW indicates that the observations are unable to accurately constrain the posterior marginal PDF about the truth solution. Note, however, that the mode in marginal
406
MONTHLY WEATHER REVIEW
VOLUME 142
FIG. 1. Posterior PDF for process parameters, radar reflectivity, and observational constraint. Parameter values for ‘‘truth’’ simulation are indicated by red crosshairs.
distributions does not necessarily correspond to the mode in the full 12-dimensional PDF. The process of marginal integration preserves the locations of modes for Gaussian distributions, but not necessarily for any other distribution type. The lack of constraint in DGACR, QRACS, PSFW, and PSACR is puzzling, given that these processes were chosen for perturbation because of the strength of their activity in the truth simulation. One possible explanation is that the chosen parameter range of 0–3 is insufficient to explore the likely range of perturbation of
these processes. Alternatively, process activity may be ‘‘fixed’’ by the prescribed dynamics and thermodynamics, limiting the sensitivity of process parameter perturbation. A simpler possibility is that the unconstrained processes are simply not as crucial for explaining the microphysical activity within the simulated storm. By contrast, the strong constraint on process parameters such as PRACW, PSDEP, and DGACW would indicate that these processes dominate the variability of simulated storm microphysics. By way of comparison, the traditional microphysical parameters of VVP12 are all
407
VAN LIER-WALQUI ET AL.
well constrained within the prior bounds. This can be interpreted as an indication that perturbation of the traditional microphysical parameters affects and constrains the activity of both more and less important processes alike. The relative lack of constraint on these process parameters does not, however, indicate that these processes have no relevance to microphysical parameterization uncertainty. Despite the near-uniformity of the distribution of DGACR, this variable clearly maintains some control of the experiment, as is shown in the joint marginal distribution with the PSACW process parameter, which displays covariance between these two variables. That is to say, while variations in DGACR are not constrained by observations, joint variations in the DGACR and PSACW process parameters are constrained. It is an unavoidable consequence of multivariate PDFs that the only relationships that can be easily analyzed are those that appear in the marginal distributions—it is likely that there exist analogous trivariate and higher-variate constraints that are simply not visible in the two-dimensional marginal distributions. These relationships may contribute importantly to the structure of the posterior forecast, even if they do not result in obvious constraint on the marginal distributions of these parameters. Joint distributions of process parameters with actual process activity, shown in Fig. 2 shed more light on the model control offered by the process parameters. The value on the ordinate corresponds to the value of each process tendency contribution, integrated along the spatial dimension of the model, as well as 10 minutes before and after each observation time (i.e., 50–70 min for the stratiform model region, and 110–130 min for the convective model region). The abscissa is the corresponding process parameter value for each member of the control parameter set. For process activity within the convective storm phase, nearly all joint marginal distributions show some degree of linear control by the process parameters on actual model process tendency. Exceptions include evaporation of rain, ERN, which shows a nonmonotonic relationship between process parameter and process activity. PSMLT, PGNLT, and PIACR parameters display some degree of linear control of their respective processes before monotonically losing sensitivity. Finally, DGACW displays considerable uncertainty in the relationship between parameter and process tendency, shown by a diffuse shape with limited covariance between parameter and process. Other parameters show linearity with varying degree of uncertainty in the relationship, from very little (PSDEP) to more substantial (QRACS). Uncertainty is representative of the exclusivity of the control on process tendency by a particular process parameter. That is, if
FIG. 2. Posterior joint marginal PDFs of process parameters with their respective process output. Process output obtained from model tendency, integrated across vertical dimension, and 610 min about (a) convective or (b) stratiform observation time, as appropriate.
JANUARY 2014
408
MONTHLY WEATHER REVIEW
VOLUME 142
FIG. 3. (top) Posterior joint marginal PDF of PSMLT process parameter with PSMLT process activity in the convective regime, as in Fig. 2. (bottom) Three-dimensional joint marginal between these variables and an additional variable, the QRACS process parameter, shown in slices, each with a different value of this third parameter.
many parameters control the value of a given process activity, the relationship of any single parameter with the process activity will have considerable variability (unless there is strong covariance between the parameters). Conversely, if the joint distribution is closely constrained to a line, this is an indication that a single parameter is the primary source of variability of that process. Note that for some processes, such as PSMLT and PIACR, the degree of uncertainty in the relationship between process parameter and process activity varies depending on the value of the parameter. That is, for low values (0–1) of the PSMLT process parameter, there is far less uncertainty in the value of PSMLT process activity compared with high values (2–3). The joint marginal distributions between process parameter and process tendency output for the stratiform storm phase show broad similarities with those from the convective region. PSMLT, PIACR, DGACR, QRACS, PSFW, PSACR, and PSACW show little change, besides slightly more or less uncertainty in the relationship between process parameter and process tendency. On the other hand, ERN, PGMLT, and PSDEP show significant change in the shape of the PDF. For ERN, the same nonmonotonic trend observed for the convective region is shown to be enhanced here with a switch to a convex function at high values of the ERN process parameter. The process parameters of PGMLT and PSDEP appear to lose control over their respective process activities in the stratiform region, as shown by the lack of covariance in the joint marginal distributions. Finally, for PRACW, the covariance between process parameter and process activity changes sign between the convective and stratiform regions. In some cases, uncertainty observed in Fig. 2 can be diagnosed by considering bivariate sensitivities (i.e., sensitivity of a process activity to simultaneous variation
of two process parameters). One such example is shown in Fig. 3 for PSMLT, with QRACS as the second process parameter. This figure demonstrates that variations in QRACS are responsible for uncertainty shown in the marginal between the PSMLT process parameter and the PSMLT process activity term. For any fixed value of QRACS, the uncertainty in this controlling relationship for PSMLT is much lower than considering all likely values of QRACS. Furthermore, the value of the QRACS process parameter appears to set a limit on PSMLT activity, with greater values of QRACS resulting in increased ‘‘clipping’’ of QRACS activity. An intuitive explanation for this behavior is that as the QRACS parameter is increased, rain accretion of snow is enhanced to the point where the amount of snow melting is limited by the exhaustion of snow hydrometeor concentrations. The clipping behavior is not unique to PSMLT, and is also shown in the joint parameter-process marginal distributions of PIACR and, to a lesser extent, QRACS, as shown in Fig. 2. Another example of bivariate control of process activity is shown in Fig. 4, for the modulation of the parameter–process relationship of PGMLT by variations in the PSACW parameter. In this case, however, the sensitivity of the activity of PGMLT to perturbations in the PGMLT process parameter is increased with increasing values of the PSACW parameter. This can be seen in the increase in slope of the PGMLT–PGMLT relationship with increasing values of PSACW. Figure 5 shows Shannon information gain over five column-integral model variables, for the posterior distribution produced by the current inverse experiment using the new process parameters. For comparison, information gain is also shown when using traditional microphysical parameters as control variables, that is, using the posterior distribution from VVP12. These results indicate that for all variables, typically more
JANUARY 2014
VAN LIER-WALQUI ET AL.
409
FIG. 4. As in Fig. 3, but for process parameter and activity of PGMLT together with variation in PSACW.
information is gained in the forecast of column-integral model variables when using the new process parameter experiment than the traditional microphysical parameters of VVP12. This is an indication of less uncertainty
in the posterior estimate of these variables when the new process parameters are used. Information gain measures a sharpening of a distribution relative to some prior distribution (in this case a bounded uniform with a range
FIG. 5. Information gain on five column-integral measures relative to uninformed prior for values at the (top) convective regime and the (bottom) stratiform regime, for the microphysical parameter inversion of VVP12 (red) and the current work (blue). Posterior distributions of these variables shown together with the value for the truth simulation.
410
MONTHLY WEATHER REVIEW
VOLUME 142
FIG. 6. Squared Mahalanobis distance, plotted against quantiles of a chi-square distribution. The dots show Mahalanobis distance for thinned samples of the posterior parameter distribution, while the dashed line is a slope 5 1 line, which represents Gaussianity. (a) Posterior distribution from VVP12, (b) posterior distribution estimated in the current work, (c) a banana-shaped distribution, (d) a lognormal distribution, and (e),(f) sample marginal PDFs from the banana and lognormal distributions, respectively.
that spans the forecast prior distribution of both microphysical parameters and process parameters); it does not indicate if the mode of that distribution displays bias relative to the ideal value of the variable. To indicate any possible forecast bias for these variables, posterior distributions are also shown in Fig. 5 for both experiments together with the value for the truth simulation. In addition to the expected reduced variance of posterior column-integral variable distributions with the new process parameters, the mode of the distribution shows
reduced bias relative to VVP12 in both the convective and stratiform storm phases. In particular, observation types sensitive to ice hydrometeors, such as ice water path and shortwave radiation show more bias for VVP12 compared with the current experiment. To determine how well the posterior parameter distribution approximates a multivariate Gaussian, Mahalanobis distance is calculated. Figure 6 shows Mahalanobis distance for thinned samples from the parameter distributions of VVP12 and the current work. If a distribution is
JANUARY 2014
VAN LIER-WALQUI ET AL.
TABLE 2. Mardia’s measures of multivariate skewness and kurtosis for the posterior parameter PDF of VVP12 and the current work.
VVP12 Current expt
Skewness
Kurtosis
22.455 6.085
146.260 164.543
a multivariate Gaussian, the Mahalanobis distance of its samples will follow a chi-square distribution, and points should fall along the dashed line when plotted against chisquare quantiles. Deviations from this line thus represent deviations from multivariate Gaussianity. It is clear that such deviations are significantly more severe for the microphysical parameter posterior PDF of VVP12 than for the current process parameter posterior PDF. Indeed, the degree to which the current work outperforms VVP12 by this metric is worthy of note, given that the posterior marginal distribution (Fig. 1) still displays skewness and indicates multimodality for some parameters. The Mahalanobis distances of numerous ‘‘standard’’ 10-dimensional distributions were calculated for comparison. The two distributions which most resemble the current study and VVP12 are shown. The first is a 10-dimensional Gaussian where two dimensions have been ‘‘twisted’’ together to produce a banana-shaped PDF. The second is a 10-dimensional lognormal distribution. These plots suggest that for the new process parameters, the primary source of non-Gaussianity is lognormal skewness, whereas the primary source of nonGaussianity in the traditional parameter PDF of VVP12 is curved, banana-shaped distributions. Whereas lognormality may be corrected by a simple log-transformation of control variables, the banana shaped distribution presents a less trivial problem. Multivariate measures of Gaussianity such as Mahalanobis distance are useful to augment analysis of the marginal distributions shown in Fig. 1, because marginal distributions are the result of an integration process in which information about the shape of the PDF and the location of its modes within the 12-dimensional parameter space is altered or lost. An analogy is to consider that the shadow of an object gives little indication of the full three-dimensional shape of that object. Likewise, the severe non-Gaussianity of the posterior in VVP12 may be an irreducibly 10-variate characteristic not easily visualized in marginal distributions. Here, the Mardia (1970) tests of multivariate skewness and kurtosis, shown in Table 2, demonstrate that while the posterior distribution in the current work outperforms VVP12 with regard to skewness, it suffers from slightly greater kurtosis—a result that is surely related to the near-uniform posterior distributions of DGACR, PSFW, and PSACR.
411
5. Conclusions New control variables that multiplicatively perturb modeled microphysical process tendencies are proposed for the estimation of microphysical parameterization uncertainty. This choice is motivated by the hypothesis that these parameters may more linearly control observable model behavior than the traditional microphysical parameters used in PV10 and VVP12. Twelve process parameters are chosen over which to perform a probabilistic inversion, following PV10 and VVP12. These parameters control the evolution of a 1D atmospheric column model with single-moment bulk microphysics parameterization. The model is forced with time-varying profiles of water vapor and vertical velocity tendency to simulate the microphysical variability typical of midlatitude squall lines. Vertically resolved simulated radar observations drawn from two model times of a ‘‘truth’’ simulation provide observation constraint. Results indicate that model evolution is not obviously sensitive to all process parameters—many parameters have a near-uniform posterior marginal PDF. Nevertheless, these unconstrained parameters display covariance with other parameters and represent a source of uncertainty in the relationship between parameters and modeled processes (see, e.g., Fig. 3). The posterior parameter PDF shows skewness, caused by monotonic nonlinearity, which is possibly related to the positivedefiniteness of the uncertain control variables. Multimodality is also present in the marginal distributions for some parameters, indicating nonmonotonic nonlinearity in the relationship between control variables and observable variables. While this sort of nonlinearity may present difficulties for Gaussian estimation techniques, further analysis of posterior distributions of model variables using the new process parameters indicates improvement relative to the traditional microphysical parameters. For example, joint distributions of process parameters and process activity (Fig. 2) indicate that within many parameter’s range of variability, parameter perturbation exerts a roughly linear control over corresponding process activity—often with consistency in the relationship between convective and stratiform storm morphologies. These results indicate that the new process parameters, with little exception, provide roughly linear control over their target processes, and thus may be useful control variables for an ensemble Kalman parameter estimation experiment. Indeed, perfect linearity in parameter control, and associated Gaussianity, is not a necessary condition for the application of Gaussian methods. Of greater concern is forecast ensemble bias, especially in the case that uncertainty is reduced by new or more precise observations.
412
MONTHLY WEATHER REVIEW
Mahalanobis distance, a multivariate measure of Gaussianity further supports the hypothesis, indicating significantly improved Gaussian characteristics of the current choice of control parameters, relative to VVP12. Mardia’s tests of multivariate skewness and kurtosis indicate that the improvement largely involves reduced skewness of the posterior in the current study, with kurtosis slightly increasing. The significant value of skewness of both distributions suggests that further improvement may result from rescaling variables under the assumption that they are log-normally distributed. This would be a more natural choice because the lognormal assumption is consistent with the nature of the current process parameters, which are multiplicative perturbations, and thus positive-definite. This hypothesis is further supported by the resemblance of the Mahalanobis distance of the new process parameters (Fig. 6b) to that of a lognormal distribution (Fig. 6d). The high kurtosis for the process parameters may be due to the truncation of distributions by the imposed prior range of 0–3. The choice of control parameters over which to estimate microphysical parameterization uncertainty can also be judged by considering the characteristics of the forecast ensemble generated by the uncertain parameter distributions. The analysis of posterior distributions of five column-integral model output variables, shown in Fig. 5, indicates that there is generally less variance for the experiment with new process parameters—a result that is corroborated by greater Shannon information gain relative to that from the traditional microphysical parameter experiment. Crucially, the posterior distributions for these model output variables display reduced bias relative to truth for the new process parameter experiment when compared with those from the traditional microphysical parameter experiment. These results are analogous to a forecast distribution and serve to confirm the new process parameters as a natural and robust choice for the estimation of microphysical parameterization uncertainty. As mentioned in section 1, the new process parameters can be thought of as a natural choice for the estimation of uncertainty which may include error in the physical assumptions of the parameterization scheme. This error can be characterized as ‘‘aleatory’’ uncertainty, or error which is an unavoidable consequence of model inability to represent reality. By contrast, if uncertainty is expected to be primarily epistemic, that is, a function of not knowing the optimal traditional parameter values, then these traditional parameters may be a more meaningful choice for uncertainty quantification. It is likely that both types of uncertainty are present in microphysical parameterization. A discussion
VOLUME 142
of partitioning uncertainty into each category is beyond the scope of the current investigation. In summary, the proposed choice of multiplicative process perturbation parameters shows promise as a basis for the estimation of microphysical parameterization uncertainty with improved linearity characteristics. Work is under way to test whether this translates to an improved inverse estimate when using Gaussian inverse techniques such as ensemble Kalman methods. If this is confirmed, it suggests that assimilation of real data into 3D models may be performed in order to estimate microphysical parameterization uncertainty. Estimation of microphysical parameterization uncertainty is one part of accounting for uncertainty in numerical weather prediction models. A full account of weather model uncertainty would not only yield improved probabilistic forecasts, but also provide crucial insight into elements of the model that require improvement. Acknowledgments. M. van Lier-Walqui was supported by NSF Grant AGS-1019184. D. Posselt was supported by NASA Modeling, Analysis and Prediction Grants NNX09AJ43G and NNX09AJ46G, as well as Office of Naval Research Grant N00173-10-1-G035.
REFERENCES Coddington, O., P. Pilewskie, and T. Vukicevic, 2012: The Shannon information content of hyperspectral shortwave cloud albedo measurements: Quantification and practical applications. J. Geophys. Res., 117, D04205, doi:10.1029/2011JD016771. Haario, H., E. Saksman, and J. Tamminen, 2001: An adaptive Metropolis algorithm. Bernoulli, 7 (2), 223–242. ——, M. Laine, and A. Mira, 2006: DRAM: Efficient adaptive MCMC. Stat. Comput., 16, 339–354. Hastings, W., 1970: Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57 (1), 97–109. Haynes, J. M., R. T. Marchand, Z. Luo, A. Bodas-Salcedo, and G. L. Stephens, 2007: A multipurpose radar simulator package: QuickBeam. Bull. Amer. Meteor. Soc., 88, 1723–1727. Hodyss, D., 2011: Ensemble state estimation for nonlinear systems using polynomial expansions in the innovation. Mon. Wea. Rev., 139, 3571–3588. Kessler, E., III, 1969: On the Distribution and Continuity of Water Substance in Atmospheric Circulations. Meteor. Monogr., No. 32, Amer. Meteor. Soc., 46 pp. Mardia, K. V., 1970: Measures of multivariate skewness and kurtosis with applications. Biometrika, 57 (3), 519–530. Metropolis, N., A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller, 1953: Equation of state calculations by fast computing machines. J. Chem. Phys., 21 (6), 1087, doi:10.1063/ 1.1699114. Mira, A., 2001: On Metropolis–Hastings algorithms with delayed rejection. Metron, 59 (3–4), 231–241. Morrison, H., G. Thompson, and V. Tatarskii, 2009: Impact of cloud microphysics on the development of trailing stratiform precipitation in a simulated squall line: Comparison of
JANUARY 2014
VAN LIER-WALQUI ET AL.
one- and two-moment schemes. Mon. Wea. Rev., 137, 991– 1007. Posselt, D. J., and T. Vukicevic, 2010: Robust characterization of model physics uncertainty for simulations of deep moist convection. Mon. Wea. Rev., 138, 1513–1535. ——, and C. H. Bishop, 2012: Nonlinear parameter estimation: Comparison of an ensemble Kalman smoother with a Markov chain Monte Carlo algorithm. Mon. Wea. Rev., 140, 1957– 1974. ——, T. S. L’Ecuyer, and G. L. Stephens, 2008: Exploring the error characteristics of thin ice cloud property retrievals using a Markov chain Monte Carlo algorithm. J. Geophys. Res., 113, D24206, doi:10.1029/2008JD010832. Shannon, C. E., 1948: A mathematical theory of communication. Bell Syst. Tech. J., 27, 379–423, 623–656. Sondergaard, T., and P. F. J. Lermusiaux, 2013: Data assimilation with Gaussian mixture models using the dynamically orthogonal field equations. Part I: Theory and scheme. Mon. Wea. Rev., 141, 1737–1760. Stensrud, D. J., J.-W. Bao, and T. T. Warner, 2000: Using initial condition and model physics perturbations in short-range ensemble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 2077–2107.
413
Tao, W.-K., and J. Simpson, 1993: Goddard Cumulus Ensemble Model. Part I: Model description. Terr. Atmos. Oceanic Sci., 4 (1), 35–72. ——, and Coauthors, 2003: Microphyics, radiation and surface processes in the Goddard Cumulus Ensemble (GCE) model. Meteor. Atmos. Phys., 82, 97–137. Tarantola, A., 2005: Inverse Problem Theory and Methods for Model Parameter Estimation. SIAM, 352 pp. Thompson, B., 1990: Multinor: A Fortran program that assists in evaluating multivariate normality. Educ. Psychol. Meas., 50, 845–848. van Leeuwen, P. J., 2010: Nonlinear data assimilation in geosciences: An extremely efficient particle filter. Quart. J. Roy. Meteor. Soc., 136, 1991–1999. van Lier-Walqui, M., T. Vukicevic, and D. J. Posselt, 2012: Quantification of cloud microphysical parameterization uncertainty using radar reflectivity. Mon. Wea. Rev., 140, 3442–3466. Vukicevic, T., and D. Posselt, 2008: Analysis of the impact of model nonlinearities in inverse problem solving. J. Atmos. Sci., 65, 2803–2823. ——, O. Coddington, and P. Pilewskie, 2010: Characterizing the retrieval of cloud properties from optical remote sensing. J. Geophys. Res., 115, D20211, doi:10.1029/2009JD012830.
