Verification region selection and data assimilation for adaptive sampling CRAIG H. BISHOP1, BRIAN J. ETHERTON2 AND SHARANYA J. MAJUMDAR3 2 – University of North Carolina Charlotte, Charlotte, North Carolina 3 – RSMAS University of Miami, Miami, Florida (Submitted to the Quarterly Journal of the Royal Meteorological Society, 16/3/2005) Keywords: Targeted Observations Ensemble Transform Kalman filter Error variance prediction THORPEX SUMMARY Adaptive or targeted observations supplement routine observations at a pre-specified targeting time. Adaptive observation locations are selected to supplement routine observations in an attempt to minimize the forecast error variance of a future target forecast within some predefined verification region (VR) at some predefined verification time. Ideally, the VR is placed in a location where unusually large forecast errors are likely. Here, we compare three methods of selecting VRs. A climatological method based on seasonal averages of forecast errors. An unconditioned method based on verification time ensemble spread and a conditioned method based on an Ensemble Transform Kalman Filter (ETKF) estimate of forecast error variance given the routine observations to be taken at the targeting time. To test the effectiveness of the three approaches, Observation System Simulation Experiments (OSSEs) on a chaotic barotropic flow were performed using an imperfect model. To test the sensitivity of our results to the type of forecast error covariance model used in the data assimilation (DA) scheme, two types of DA schemes were tested: An isotropic DA scheme and a hybrid DA scheme. For isotropic DA, correlations between vorticity forecast errors at any two points were solely a function of the distance between the points. For hybrid DA, the forecast error covariance matrix was a linear combination of the covariance matrix used in isotropic DA and the sample covariance matrix of an ETKF ensemble. For each of the three VRs, the ETKF was used to select two adaptive observations and the errors of forecasts that used these adaptive observations were compared against those that didn’t. To assess targeted observation induced error reductions, forecast errors with and without targeted observations were computed for the VR, the total domain and also an empirical VR which was defined to be the VR with the largest forecast error without targeted observations. The sensitivity of the ETKF’s ability to distinguish high impact targeted observations from low impact observations to the type of DA scheme employed was also examined. Amongst other things, it was found that (a) conditioned VRs were more prone to large errors than the other VRs, and (b) the targeted observation induced reduction in forecast error variance was largest when hybrid DA and conditioned VRs were used as was the range of forecast impacts distinguished by ETKF predictions.
1. INTRODUCTION The World Meteorological Organization’s (WMO) THORPEX research program has called for the development of a Globally Integrated Forecasting System (GIFS) that would allow the observing system, observations, assimilation and the forecast model to be adaptively configured to maximize forecast skills for specific societal and economic uses (see, http://www.wmo.int/thorpex/pdf/brochure.pdf). One of the ideas that has motivated this call for a GIFS is the notion that unusually large forecast errors that we shall hereafter refer to as forecast busts might be mitigated by the assimilation of observations deployed in carefully selected “target” sites whose location varies according to the flow and the location of some pre-specified VR in which one seeks to improve the forecast. For this “bust mitigation” goal of targeted 1
Craig H. Bishop, Naval Research Laboratory, Marine Meteorology Division, 7 Grace Hopper Ave, Stop 2, Building 702, Room 212, Monterey, CA 93943-5502. E-mail:
[email protected]
1
observations to be realized, the location of the VR needs to adapt to predictions of the likely error in the target forecast without targeted observations. Adapting the VR location in this way ought to increase the chances that the VR would experience a forecast bust if targeted observations were not taken. Since forecast busts are a function of error growth in non-linear model forecasts and geographical variations in the quality of the routine observational network, both of these factors ought to be accounted for when selecting VRs for the purpose of bust mitigation. In the Fronts and Atlantic Storm Track Experiment (Joly et al., 1999), the North Pacific Experiment (Langland et al., 1999), the Winter Storm Reconnaissance Program (WSR) (Szunyogh et al., 2000, 2002; Majumdar et al., 2001, 2002) and the 2003 Atlantic THORPEX Regional Campaign (ATRec) (Mansfield et al., 2005) VRs and times were generally subjectively selected by forecasters to correspond to regions over countries of interest for which a forecast of significant weather was accompanied by significant ensemble spread. There are two concerns with this approach. First, the ensemble spread maps used in these experiments indicate the uncertainty of the latest forecast. They do not indicate the uncertainty of the target forecast to be initialized at the targeting time. In particular, these maps do not account for the possibility that the regions where changes in the analysis can affect the target forecast may be well sampled by the routine observational network. Second, restricting the VR to localized geopolitical regions means that opportunities to reduce very large forecast errors external to these regions will be missed. On occasions, reducing large forecast errors outside of the localized geopolitical region may be of greater value to the financial supporters of adaptive observing platforms than reducing relatively small errors within a localized geopolitical region. The restriction of VRs to confined geopolitical regions and the lack of an objective technique to account for the role of routine observations in atmospheric adaptive sampling experiments may be partially responsible for the frequency with which these experiments have found that the target forecast was relatively accurate with or without the assimilation of the targeted observations (David Richardson, Zoltan Toth - personal communications). Situating VRs in locations where the chances of forecast busts were greatest would increase the potential for targeted observations to increase forecast accuracy and, in theory, reduce the frequency of forecast busts. The advent of atmospheric and oceanic observing technologies that require guidance such as underwater gliders (Rudnick et al. 2004) and proposed directable space-based lidars make the question of how to select VRs even more pertinent. One can imagine, for example, space-borne lidar or underwater gliders being focussed on target regions where additional observational scrutiny would reduce the chance of a forecast bust in VRs whose locations were selected solely on the basis of the likelihood of a forecast bust. VR selection methods and adaptive sampling methods might also become important in deciding how to thin overwhelmingly large volumes of satellite data. The real-time application of the ETKF (Bishop et al., 2001) to select aircraftborne dropwindsonde deployments in annual Winter Storm Reconnaissance (WSR) Programs (Szunyogh et al. 2000, 2002) to improve 1-3 day forecasts is described in Majumdar et al. (2002). These WSR programs have been part of National Weather Service (NWS) operations from 2001 to the present day. The ETKF was also used by the Met Office and the National Centers for Environmental Prediction (NCEP) to target observations in the recent ATReC (Mansfield et al., 2005). Toth et al. (1999, 2001, 2002) report that metrics based on surface pressure indicate that the assimilation
2
of targeted observations in the WSR campaigns has led to a 10%-25% reduction in root mean square surface pressure error in the most affected areas. This reduction is comparable to a 12-24 hour gain in surface pressure forecast accuracy. Could larger forecast improvements be obtained with an improved VR selection method? As far as the authors are aware, the idealized experiments described in this paper are the first to (a) show how the ETKF can be used to select conditioned VRs (b) compare the ability of differing objective methods to select VRs in which forecast busts are likely (c) document the sensitivity of targeted observation induced error reductions to the VR selection method used, and (d) quantify the dependencies of (b) and (c) to the DA scheme and the ensemble size. Another original aspect of this paper is its assessment of the influence of the type of DA scheme used on the ability of the ETKF to discriminate between high and low impact observations. The ETKF is one of numerous objective adaptive sampling techniques that have been designed and tested over the past decade. Examples of adaptive sampling techniques include sensitivity to changes in the analysis based on adjoint models (Bergot et al. 1999), the quasi-linear inverse method (Pu et al. 1999), rapid growth of analysis errors into a forecast VR using total energy singular vectors (Palmer et al. 1998, Buizza and Montani 1999, Gelaro et al., 1999) analysis error variance singular vectors (Gelaro et al. 2002), Hessian singular vectors (Leutbecher 2003), sensitivity to observations (Baker and Daley 2000, Doerenbecher and Bergot, 2001), and ensemble variance or “spread” (Lorenz and Emanuel 1998, Morss et al. 2001). Bachmayer and Leonard (2002) discuss using schools of autonomous underwater robots to adaptively track gradient features in the ocean. Additionally, techniques based on Kalman Filters (Bishop et al. 2001, Lermusiaux 2001, Bergot and Doerenbecher 2002, Hamill and Snyder 2002, Leutbecher 2003) can provide quantitative predictions of the covariance of forecast changes and the reduction in forecast error variance provided by adaptive observations. The performance of the hybrid and isotropic DA schemes used in this paper under two different types of model error is described in Etherton and Bishop (2004) (hereafter EB). Details of the differences between our hybrid DA scheme and that described in Hamill and Snyder (2000) are given in Section 2. Section 2 also describes the adaptive OSSEs and our methods for selecting VRs. Section 3 gives the results. Conclusions follow in Section 4. 2. ADAPTIVE OBSERVATION SYSTEM SIMULATION EXPERIMENTS As in EB and Bishop et al. (2001), a fictitious forecast agency is charged with forecasting the state of a two-dimensional turbulent flow generated using a “truth run” of a doubly periodic barotropic vorticity model. Instability and chaos within the “truth run” flow is maintained by constantly relaxing the zonally averaged flow back to a zonally invariant barotropically unstable state featuring two ribbons of high vorticity, one crudely representing the mid-latitude jet stream and the other the Inter Tropical Convergence Zone (ITCZ). A snapshot of the truth run is shown in Figure 1. The black dots on Fig. 1 indicate the locations of the routine error prone vorticity observations that the forecasting agency will use in its attempts to estimate current and future states of the turbulent flow. Each observation is created by adding a normally distributed random number to the value of the truth run at the observation site (see EB for details). Note the relative scarcity of observations on the right half of the domain. Attempts by the hypothetical forecasting agency to forecast this state using DA and a forecast model with 32x32 grid points and 200 km horizontal spacing are hampered by the fact that the forecasting agency’s model has an incorrectly prescribed
3
relaxation term. Instead of relaxing back to the time-invariant barotropically unstable state that the truth run relaxes toward, the agency model relaxes toward a zonal state whose meridional vorticity distribution is given by the left most grid points of the model at the beginning of each agency forecast. Consequently, while the state relaxed to by the truth run is always the same; the state relaxed to by the agency forecast model varies randomly from forecast to forecast and generally has weaker vorticity gradients than the state relaxed to by the truth run (again, see EB for details). The agency DA cycle is started by (a) initializing the model with the barotropically unstable zonal state plus a very small amount of grid point noise (b) waiting until vortices appear along the “southern” vorticity strip before initiating the forecast/DA cycle (b) creating an initial error in the agency’s forecast/DA cycle by letting the state used to initialize the agency forecast/DA cycle lag the true state by 24 hrs, and (c) giving the forecast/DA cycle a 50 day spin-up period before beginning the experiments (see EB for further detail). Following the 50 day spin-up period, 9 independent but otherwise identical 250 day OSSEs are performed. The independence of the 9 250 day OSSEs is imparted by using independent seeds for the strings of random numbers used to generate observation errors during the experiment and also during the last 10 days of the spin-up period. Performing 9 independent but otherwise identical OSSEs allows us to infer the statistical significance of our results.
Figure 1 – The vorticity field of the pseudo-truth run on day 1, and routine observational network (dots). Vorticity is in units of 10-5 s-1, and the contour interval is 0.1x10-5 s-1.
Over each 250-day OSSE, daily 24 and 48-hour forecasts of the state of the turbulent flow are made. As in EB, observations of vorticity are taken from the truth run every 24 hours and combined with the 24-hour agency forecast valid at the observing time to produce a new analysis. Given the relatively slow error growth rates in the barotropic model, the temporal spacing of 24 hrs between observation times crudely mimics the ratio of error doubling time (24 hrs) to analysis period (6 hrs) found at most numerical weather prediction (NWP) centers. The simple targeting problems addressed in this paper are that of using a 48 hour ensemble forecast initialized at t = ti to (a) choose a VR and (b) decide where two targeted observations should be placed at t = ti + 24 in order to improve the
4
verification time t = ti + 48 forecast initialized at t = ti + 24 within the VR. (a) Isotropic and hybrid error covariance model As in EB, for the isotropic error covariance model the covariance cij between vorticity errors at the ith and jth grid points is given by
2 2 cij = A ⎡⎢1 + ln(0.1) ( rij / D ) ⎤⎥ exp ⎡⎢ln(0.1) ( rij / D ) ⎤⎥ . (1) ⎣ ⎦ ⎣ ⎦ where D defines the correlation length scale, rij gives the separation distance between
the ith and jth gridpoint and the constant A gives the forecast error variance. Note that this function precisely describes the structure of the analysis increment that would be obtained from an observation of vorticity at the ith grid-point using the isotropic error covariance model. This particular vorticity structure is proportional to the Laplacian
(
) ⎥⎦
2 of exp ⎡ ln(0.1) rij / D ⎤ and hence is consistent with an exponential correlation
⎢⎣
model for the streamfunction. The factor ln(0.1) simply ensures that the streamfunction error correlation between grid points separated by the distance D is equal to 0.1; in addition, the factor ln(0.1) is less than zero and hence ensures that correlations between grid points decrease with separation distance. For the experiments reported on in this paper, A and D were independently tuned to minimize 24-hour forecast error variance over a period encompassing the 50 day spin up period and the 250 day OSSE. The tuning was performed by testing many different values over reasonable ranges that contained minima. We shall denote the error covariance f . matrix associated with our application of (1) with the symbol Biso Note that while most NWP centers assume isotropic error correlations, they do not assume anisotropic error covariances whereas our highly simplified error covariance model does. We assumed isotropic error covariances for simplicity and to minimize the number of tunable parameters in our covariance model. One can argue f is qualitatively worse than the stationary error covariance models used at that Biso NWP centers because it excludes the possibility of anisotropic covariances. But since the real atmosphere has diurnal and seasonal variability and a wide variety of instability types (baroclinic, barotropic, symmetric, convective, etc) whereas our barotropic truth run only supports barotropic instability and has no diurnal or seasonal variability, one can also argue that the assumption of stationary forecast error covariances is much better for our barotropic equation model than it is for the real atmosphere. Because of these uncertainties, there are no guarantees that the results of our OSSEs will be reproducible in operational models. Our hybrid forecast error covariance model at the time ti takes the form
(
f B hyb ti | H tri − 24
(
where P f ti | H tri − 24
)
(
)
f = (1- α ) ⎡ ρ P f ti | H tri − 24 ⎤ + α ( λ Biso ), ⎣ ⎦
(2)
) is the sample covariance matrix of the ensemble forecast that
was initialized at the time ti − 24 hrs using routine observations at time ti . In other words, it is the ensemble covariance at ti given routine observations 24 hours earlier. Note that no attempt is made to localize ensemble covariances via Schur products neither in the ETKF ensemble generation scheme nor in eq. (2). Throughout this
(
paper, a bracketed term of the form t1 | H t2
) following any matrix or vector symbol
indicates that the entity represents an estimate of some aspect of the atmospheric state
5
at the time t1 , given observations up to and including those taken at the time t2 . The observation operator H t2 maps state vectors to the variables observed at the time t2 . Superscripts r, a, and r+a indicate whether the vector or matrix superscripted respectively pertains to the routine, the adaptive, or the routine and adaptive components of the observational network. The parameters ρ and α in (2) have two purposes. First, ρ and α ensure
(
that the traces of ρ H tri P f ti | H tri − 24
)( H ) r ti
T
( )
f and λ H tri B iso H tri
T
are identical. EB
modified Hamill and Snyder’s (2000) original hybrid formulation to include this feature after realizing that without it, the weight α does not represent the amount of forecast error variance in observation space attributable to the isotropic covariance
(
model in situations where the trace of H tri P f ti | H tri − 24
( )
)( H ) r ti
T
is not equal to the
T
f trace of H tri B iso H tri
. The second purpose of these parameters is to ensure that a 5
day average of the squared magnitude of the difference between a forecast of the
(
observed state H tri x f ti | H tri − 24
)
and the observed state y tri is equal to the trace of
the sum of the hybrid error covariance and observation error covariance matrices in observation space. This second purpose is motivated by the fact that if the DA scheme is optimal
( y − Hx ) f
2
= Trace ( HP f HT + R ) where the angle brackets indicate
the mean or expected value of the quantity concerned. This approach is similar to that used in EB where more details of the procedure can be found. The difference of this approach from that used in EB is that here a 5-day average of squared innovation vectors was used whereas in EB only the current innovation vector was used. The change was made because theory indicated that estimates based on a single innovation were highly prone to sampling error for our relatively small observational network and because tests demonstrated that the averaging approach led to a superior forecast and analysis scheme. As in EB, an ETKF spherical simplex ensemble generation scheme equivalent to that described in Wang et al. (2004) was used to generate the ensemble employed in this paper. The question of the size of the ensemble to be used in our OSSEs experiments is worth discussion. Currently, the largest ensemble being run operationally has 50 members2 while the order of the number of variables and/or observations is between 105 and 107 . In our OSSEs, the forecast model has 32x32=1024 independent variables and there are 72-74 observations. Consequently, even with a 2 member ensemble, the ratio of the number of ensemble members to number of observations or model variables would still be much greater than that used in operations. Reducing the ensemble size reduces ETKF performance not just because the ratio of ensemble size to observation and variable count decreases but also because variance estimates become unreliable as the sample size drops. For example for a 64 member ensemble, the Chi-Square distribution indicates that the sample variance would lie between 78% and 123% of the true variance in 4 out of 5 trials for a perfect random sample 2
A 50 member ensemble is currently run at the European Center for Medium Range Weather Forecasting (ECMWF) (see Molteni et al. 1996),
6
ensemble. However with 16 members, the sample variance would lie between 57% and 149% of the true variance in only 4 out of 5 trials for a perfect random sample ensemble. From this analysis, one can see that our low-dimensional model makes it impossible for us to maintain the accuracy of the error variance estimates one can obtain with ensembles of about 50 members while reducing the our ratio of ensemble size to observation count to something more like what is found in operational systems. Nevertheless, the sensitivity of ensemble-based targeting and DA techniques is of fundamental importance. Consequently, we perform both 64 and 16 member experiments. If B and R were equal to the true forecast and observation error covariances, respectively, and H mapped the model space to observation space, then the minimum error variance analysis would be given by
x a = x f + BHT ( HBHT + R )
−1
( y − Hx ) f
(3)
(see Daley 1991). In this study, B is imprecisely known and (3) does not give the f we get minimum error variance estimate. When the B in (3) is set equal to B hyb
isotropic DA when α =1 and hybrid DA when α =0.5. Throughout this paper, the matrix inverse indicated in (3) is evaluated using a direct non-variational technique. For all the experiments discussed in this paper, vorticity observation errors were assumed to be uncorrelated with a variance of 2 x10−14 s −2 ; i.e. R= 2 x10−14 I where I is the identity matrix. Also note that in this study the ensemble mean K
x f = (1/ K ) ∑ xif is used as the first guess field in all DA schemes employed. Thus, i =1
in our implementation of isotropic DA the analyses and the ensemble are not entirely independent because even though the covariance matrix B is entirely independent of the ensemble with isotropic DA the first guess field is given by the ensemble mean. We chose to do this because we found that the ensemble mean 24 hr mean square forecast error was consistently smaller than that of the corresponding unperturbed forecast and we wanted to ensure that the differences in the skills of isotropic and hybrid DA could not be partially attributable to using or not using an ensemble mean as the first guess. This differs from current practice at most operational centers where the first guess field for quasi-isotropic DA is a high resolution control forecast that is entirely independent of the ensemble. For each combination of DA scheme and ensemble size, the required sets of initial conditions for the 48 hr control forecasts and 48 hr ensemble forecast were obtained from 250 day base runs in which only routine observations were assimilated. The 250 sets of 48 hr ensemble forecasts from the base runs were then used to select VRs at 48 hrs using either the climatological, the unconditioned, or the conditioned election VR method (see below). Having selected the VRs, the ETKF was then applied to the ensemble to select targeted observation sites likely to reduce forecast busts in the selected VRs (see below). To see the impact of the targeted observations on, two analyses were made at the targeting time: one that assimilated just routine observations and another that assimilated both routine and targeted observations. The differences between the accuracies of the two 24 hr forecasts made from these two analyses were then used to assess the impact and value of the targeted observations. For all experiments, the signal or change in the forecast resulting from the targeted observations together with the error of the forecast with and without targeted observations was computed and stored. Note that the accuracy of the error covariance model used to assimilate
7
observations affects the error reducing potential of targeted observations in two competing ways. On the one hand, a relatively accurate error covariance model makes better use of routine observations and hence reduces the potential of targeted observations to improve forecasts. On the other hand, a relatively accurate error covariance model can make better use of targeted observations in cases where the routine network fails to preclude large errors. (b) Selection of verification regions (VRs) Before choosing adaptive observation sites, a (1600 km)2 VR is selected, corresponding to an 8 by 8 square of model grid boxes. The climatological method places the VR over the 8 by 8 square with the largest seasonally averaged forecast error variance when only routine observations are assimilated. The unconditioned method of VR selection examines the 48 hr ensemble variance given by the diagonal
(
terms of P f ti + 48 | H tri
) to locate the particular 8 by 8 square of model grid boxes
in which the sum of the 48 hr ensemble variances of vorticity at each of the 64 grid points within the VR is maximized. This approach has similarities with that currently used in NCEP’s WSR program in which VRs are selected by an experienced forecaster on the basis of raw ensemble spread and subjective judgments on the potential socio-economic cost of an erroneous forecast of a high-impact weather event. The deficiency of this approach is that the effect of the routine observations on the accuracy of the forecast is not explicitly taken into account. In the following, we describe a conditioned method of VR selection that accounts for the effect of the assimilation of routine observations on forecast value. In our prototype simple system, forecast value is equated with forecast accuracy. If transitioned to operations, the definition of forecast value used would, no doubt, have a dependency on socioeconomic impact as judged by the controllers of the adaptive observational platform. Following Bishop et al. (2001), this distribution of forecast error variance is
(
obtained by transforming the square root Z f ti + 48 | H tri
(
forecast error covariance matrix P f ti + 48 | H tri
)
)
of the ensemble based
that does not account for routine
(
)
observations at the targeting time into the square root Z f ti + 48 | H tri + 24 of the
(
forecast error covariance matrix P f ti + 48 | H tri + 24
)
that does account for routine
(
)
observations at the targeting time. The square root of P f ti + 48 | H tri used is the
(
)
nxK matrix Z f ti + 48 | H tri whose jth column
{Z (t + 48 | H )} f
r ti
i
= j
(
) (
)
1 ⎡ f x j ti + 48 | H tri − x ti + 48 | H tri ⎤⎦ ⎣ K −1
(4)
(
lists the jth ensemble perturbation of the 48 hr ensemble forecast x fj ti + 48 | H tri
(
about the ensemble mean x ti + 48 | H tri
(
ETKF transformation of Z f ti + 48 | H tri
(
)
)
divided by the square root of K-1. The
) into the square root Z ( t + 48 | H ) of f
r ti + 24
i
P f ti + 48 | H tri + 24 is given by
(
)
)
(
) (
P f ti + 48 | H tri + 24 = Z f ti + 48 | H tri + 24 Z f ti + 48 | H tri + 24
8
)
T
(5)
and
(
)
(
)
Z f ti + 48 | H tri + 24 = Z f ti + 48 | H tri Cr ( Γ r + I )
−1/ 2
,
(6)
where Cr and Γ r are orthogonal and diagonal matrices, respectively, obtained from the eigenvector decomposition
⎡ H tr + 24 Z f ( ti + 24 | H tr ) ⎤ i ⎦ ⎣ i
T
(R ) r ti + 24
−1
⎡ H tr + 24 Z f ( ti + 24 | H tr ) ⎤ = Cr Γ r Cr T i ⎦ ⎣ i
(7)
As shown in Bishop et al. (2001), this transformation ensures that at the targeted observation time tt = ti + 24 , the well-known error covariance update equation (Daley, 2001) for an optimal DA scheme
(
)
(
)
(
P f ti + 24 | H tri + 24 = P f ti + 24 | H tri − P f ti + 24 | H tri
(
× ⎡ H tri + 24 P f ti + 24 | H tri ⎣⎢
(
)(
H tri + 24
)
T
−1
(
)( H ) r ti + 24
T
+ R tri + 24 ⎤ H tri + 24 P f ti + 24 | H tri ⎦⎥
)
(8)
)
is satisfied. The P f ti + 48 | H tri + 24 given by (6) may be viewed as a quasi-linear
(
)
propagation of P f ti + 24 | H tri + 24 , (Bishop et al., 2001).
(
The diagonal terms of P f ti + 48 | H tri + 24
) give the ETKF prediction of the
24-hour forecast error variance given routine observations at the targeting time. This estimate accounts for the fact that routine observations will reduce forecast errors that originate over the highly scrutinized western half of our domain. The final step in this second method is to compute the sum of the 64 diagonal terms of
P f ( ti + 48 | H tri + 24 ) corresponding to the vorticity in every 8 by 8 square of model
grid boxes in our domain. The 8 by 8 square for which this sum is maximized is then selected as the conditioned VR. (c) Selecting targeted observation sites given the routine network Since the left half of our domain is well scrutinized by the routine observational network (Figure 1), we only search the right half of the domain for effective placements of the two adaptive observations. Furthermore, we assume that the adaptive observations will be collocated with grid points. For n possible sites and p adaptive observations at p different sites, there are
n! possible ( n − p )! p !
configurations of the adaptive observational network. In the experiment considered here (n=512 , p=2), there are over 100,000 feasible adaptive observational networks to consider on the right side of the domain. One approach would be to use the ETKF to find which of these 100,000 networks would minimize forecast error variance within the VR for each of the 250 days in each case. We chose not to use this approach because of the high computational cost. Instead, we employed the serial targeting technique described in both Bishop et al. (2001) and Majumdar et al. (2002). In this approach, the first site is obtained by using the ETKF to find the site of a single additional observation that minimizes the ETKF prediction of forecast error variance in the VR given the routine observational network. The second site is the one that most reduces forecast error variance in the VR given the routine observational network and the first adaptive observation. With this approach, one only needs to
9
compute the forecast error variance associated with about nxp (~1000) networks. Thus, in the current example it is two orders of magnitude more computationally efficient than assessing the value of all feasible networks. To identify the best location for a single additional observation, the ETKF predicts the signal variance in the VR at the verification time for every feasible location of the additional observation. The “best” location is deemed to be the location that maximizes signal variance in the VR. As shown in Bishop et al. (2001), for an optimal DA scheme, the signal variance is precisely equal to the reduction in forecast error variance. Recall that the signal variance due to an observation is the expected squared difference between the forecast that uses the observation and the forecast that doesn’t. ETKF predictions of the signal variance due to a single additional adaptive observation at the m-th feasible location is given by the sum of the terms
(
r +a
corresponding to the VR on the diagonal of the covariance matrix S m ti + 48 | H i + 24m
)
of signals produced by a single observation at the m-th feasible location. As shown in Bishop et al. (2001),
(
)
( ) ( t + 48 | H ) C
(
S m ti + 48 | H tri ++a24m = ⎡ Z f ti + 48 | H tri + 24 Cam ⎤ Γ am Γ am + I ⎣ ⎦
× ⎡⎣ Z f
i
r ti + 24
am
⎤ ⎦
T
)
−1
,
(9)
where C m and Γ m are KxK orthogonal and diagonal matrices, respectively, where K is the number of ensemble members. These matrices are defined by the eigenvector decomposition: a
a
(
) ( ) T
)
−1
⎡ H tam+ 24 Z f ti + 24 | H tr + 24 ⎤ R tam+ 24 i i ⎣ i ⎦ × ⎡⎣ H tai m+ 24 Z f ti + 24 | H tri + 24 ⎤⎦ = Cam Γ am Cam T
(
(10)
a
In eq. (10), H ti m+ 24 gives the observation operator mapping the state vector to a single adaptive observation site at the m-th feasible observational site. In summary, the site for the first targeted observation is obtained by using (9) and (10) to find the best location for an additional observation given the routine network. The site for the second targeted observation is obtained by replacing
(
)
(
Z f ti + 48 | H tri + 24 by Z f ti + 48 | H tri ++a24
First
) in (9) and (10) where H
r + a First ti + 24
indicates
the routine network plus the first targeted observation. One then uses (9) and (10) to find the best site for a single observation given the routine network and the first adaptive observation. We refer to adaptive observational networks obtained in this way as serially optimized networks. Note that such networks will not be as effective as networks obtained by assessing every possible combination of adaptive observations. 3. RESULTS (a) Verification Region (VR) Selection A primary measure of the usefulness of a VR selection method is the magnitude of the forecast errors that occur within it when no targeted observations are assimilated. Fig. 2 (for hybrid DA) and Fig. 3 (for isotropic DA) show that when no targeted observations were assimilated, conditioned VRs contained larger forecast errors than unconditioned VRs and unconditioned VRs contained larger forecast errors than climatological VRs for both 16 member and 64 member ensembles. The results plotted in Fig’s 2 and 3 represent the average of 9 independent 250 day experiments. The ordering of mean VR forecast error without targeted observations
10
shown in Fig’s 2 and 3 was found in all 9 independent experiments and the magnitudes of the averaged errors was similar for both the 64 and 16 member ensemble cases.
(x10-12s-2) Mean square 24 h fcst error in VRs (Hybrid DA) 2.5 2 1.5
Climatological
0.5
Conditioned 16 64
16 64
16 64
1
Unconditioned
16
16
16
64
64
64
6416
6416
6416
Ta rg et ed 2 O Ta bs rg et Re ed du O bs ct io n in M SE W it h
No
Ta rg et ed 2 O Ta bs rg et Re ed du O bs ct io n in M SE W it h
No
W it h
No
Ta rg et ed 2 O Ta bs rg et Re ed du O bs ct io n in M SE
0
Figure 2. The average of 9 independent 250 day averages of mean square 24 hr vorticity forecast error (MSE) in the verification region (VR) with no targeted observations, with 2 targeted observations and the difference between the two MSEs for climatological VRs, unconditioned VRs and conditioned VRs with hybrid DA. “Reduction in MSE” refers to the difference between MSE with and without targeted observations. The number above each vertical bar indicates the number of ensemble members associated with the result.
A second measure of the usefulness of a VR selection method is the forecast error reduction within the selected VR due to the assimilation of targeted observations chosen to minimize forecast error within the selected VR. Fig. 2 (for hybrid DA) again shows that under this measure the conditioned VR is better than the unconditioned VR and the unconditioned VR is better than the climatological VR. The result held for all 9 independent 250 day experiments performed with Hybrid DA for both 64 and 16 member ensemble sizes. Note that Fig. 2 shows that mean square forecast errors with no targeted observations were larger for 16-ensemble member hybrid DA than for 64 ensemble member hybrid DA. Consequently, 16-ensemble member hybrid DA, the routine observations left more room for the targeted observations to improve the forecast than with 64-ensemble member hybrid DA. In spite of this, 64-ensemble member hybrid DA was still able to extract a larger mean square error reduction from 2 targeted observations than 16-ensemble member hybrid DA. Fig. 3 (for isotropic DA) indicates that the isotropic DA results are qualitatively similar to the results for Hybrid DA. However, careful comparison of Fig’s 2 and 3 shows that the margin by which error reduction with conditioned VRs exceeded that with unconditioned VRs was smaller with isotropic DA than it was with Hybrid DA. Nevertheless, the VR error reductions due to targeted observations with conditioned VRs and isotropic DA were still found to be superior to those with
11
unconditioned VRs in 8 of the 9 independent repetitions of the 250 day trial for both 16 member and 64 member ensembles. (Recall that with Isotropic DA, ensembles are used (a) for VR selection (b) adaptive observation selection, and (c) producing the first guess field (the ensemble mean) for DA). If there were no difference in the quality of these 2 VRs, then the frequency ρ with which 9 independent repetitions of the experiment yielded 8 cases where the conditioned VRs were better than the
(
)
unconditioned VRs would be given by ρ = 9 0.59 = 0.018 . Thus, we can reject the null hypothesis that the conditioned VR is no better than the unconditioned VR with a high degree of confidence for both 16 member and 64 member ensembles. Fig’s 2 and 3 also show that with both isotropic and hybrid DA, the relative improvement one obtains on moving from climatological VRs to unconditioned VRs exceeds the relative improvement one obtains on moving from unconditioned to conditioned VRs.
(x10-12s-2) Mean square 24 h fcst error in VRs (Isotropic DA) 2.5
Climatological
Unconditioned
2 6416 1.5
64 Conditioned 16
64 16
6416
6416
6416
1 0.5
64 16
64 16
6416
Ta rg et ed 2 O Ta bs rg et Re ed du O bs ct io n in M SE W it h
No
Ta rg et ed 2 O Ta bs rg et Re ed du O bs ct io n in M SE W it h
No
W it h
No
Ta rg et ed 2 O Ta bs rg et Re ed du O bs ct io n in M SE
0
Figure 3. As in Fig. 2, but for isotropic DA.
Comparing Fig’s 2 and 3 shows that VR errors with no targeted observations were much smaller for hybrid DA than for isotropic DA. This means that the magnitude of possible forecast improvements due to targeted observations was much larger for isotropic DA than for hybrid DA. Nevertheless, the VR error reductions due to targeted observations were about the same size for hybrid DA as they were for isotropic DA. This again shows that the reduction of routine forecast errors associated with improved DA does not necessarily diminish the ability of targeted observations to reduce errors.
12
(x10-12s-2)
Mean Square 24 h fcst error in empirical VR (Hybrid DA)
3.5
I16 I64
3
I16
H16 2.5 2
I16
I64
I16
I64
H16
I64
H16
H16
H64 H64
1.5
H64
H64
1 0.5 0 No Targeted Obs
Climo VR
Unconditioned VR
Conditioned VR
Fig. 4: The mean square 24 hr vorticity forecast error within empirical VRs as a function of VR selection method for hybrid DA with 64 ensemble members (H64), hybrid DA with 16 ensemble members (H16), isotropic DA with 64 ensemble members for targeting (I64), and isotropic DA with 16 members for targeting (I16).
The 8x8 grid square that suffers the largest average forecast error without targeted observations is easily identified by examining the forecast errors of the forecasts that do not use targeted observations. We refer to these 8x8 grid squares that contain the largest errors as empirical VRs. A third measure of the usefulness of a VR selection method is the forecast error reduction within empirical VRs due to the targeted observations associated with the VR selection method. The forecast error reduction within empirical VRs as a function of VR selection method is shown for hybrid DA and isotropic DA in Fig. 4. Comparison of Fig 4 with Fig’s 2 and 3 shows that although targeted observations reduce mean square error in empirical VRs by a significant amount, the seasonally averaged error reduction is relatively insensitive to the type of VR selection method used for both hybrid and isotropic DA. The error reductions due to targeted observations within empirical VRs with hybrid DA and a 64 member ensemble (a) were greater for unconditioned VRs than climatological VRs in 8 out of 9 independent cases, and (b) were greater for conditioned VRs than unconditioned VRs in 8 out of 9 cases. Thus, we can again reject the null hypotheses that there are no differences between the VR selection methods under this measure with a high degree of confidence. However, with a 16 member ensemble and Hybrid DA the differences between the VR selection methods were not statistically significant. With isotropic DA and a 64 member ensemble, error reductions due to targeted observations were (a) greater for unconditional VRs than climatological VRs in 9 out of 9 cases (b) greater for conditioned VRs than unconditioned VRs in 5 out of 9 cases. Thus, under the reduction of error in empirical VRs measure and with isotropic DA, conditioned and unconditioned VRs are equally superior to the climatological VRs for a 64 member ensemble. However, for a 16 member ensemble the differences between the results for all 3 VR selection methods were very small and statistically insignificant. The sensitivity to VR selection method of the domain averaged error
13
reduction due to targeted observations is also of interest. Fig. 5 shows this sensitivity for hybrid and isotropic DA, respectively.
Mean square 24 h fcst error over entire grid
(x10-12s-2) 2.5
2
1.5
1
I64 I16
I64 I16
H16 H64 0.5
I64 I16
H16
I64 I16
H16
H64
H16
H64
H64
Unconditioned VR
Conditioned VR
0 No Targeted Obs
Climatological VR
Fig. 5: As in Fig. 4, but for errors averaged over the entire grid. (Note that the differences in I64 and I16 are due to the use of the ensemble mean as the first guess field in isotropic DA. These differences are statistically insignificant.)
The most striking aspect of Fig. 5 in comparison to Fig’s 2-4 is that the percent reduction in mean square forecast error due to targeted observations over the entire domain is very small in comparison to the error reductions in all of the error prone, localized VRs considered. The percentage error reductions seen in these figures range from 4% for climatological VRs, isotropic DA and a 16 member ensemble to 9% for conditioned VRs, hybrid DA and a 64 member ensemble. Although these gains may seem modest they are greater than the 3% increase in the number of observations associated with the addition of the 2 targeted observations to the 72 routine observations. Another point of interest is as follows. Figure 5 shows that the skill of the forecasts using isotropic DA has little dependence on ensemble size when averaged over the whole domain (recall that the mean of the ETKF ensemble is used as the first guess field for isotropic DA). In contrast, Fig. 4 shows that within empirical VRs isotropic DA gives significantly smaller errors when the mean of a 64 member ensemble is used as the first guess than it does when the mean of a 16 member ensemble is used. We are unaware of any compelling explanation for this fact. (b) Utility of ETKF predictions of signal variance The conditioned VRs considered in Subsection 3a were obtained from an ETKF estimate of forecast error variance given the routine network. Apart from VR selection, the ETKF was also used to identify targeted observations likely to reduce error in the selected VR. As discussed in Section 2, the ETKF does this by attempting to predict the propagation of the variance of the distribution of forecast changes due to targeted observations or signal variance. The primary indicator of the utility or discernment of a signal variance prediction scheme is its ability to distinguish occasions where the variance of the distribution of possible signals is unusually large
14
from occasions when this variance is unusually small. To assess the ability of the ETKF to do this, we first defined the climatological signal variance to be the average over each of the 64 grid points within each VR and over the entire 250 day trial period (64x250) occurrences in total) of the square of the difference between forecasts that used and did not use targeted observations. The second step was to normalize each difference or signal by the square root of its climatological signal variance. Note that the climatological average value of the square of the normalized variables is unity. For consistency, we also normalized each of the ETKF grid point predictions of signal variance (see eq. 9) within the VR by their climatological average. For the sake of brevity, in this subsection, we only consider conditioned and unconditioned VRs. The third step was to pair each normalized ETKF signal variance prediction at each grid point within the VR with the normalized difference at this grid point between forecasts that used and did not use targeted observations. The entire 250 day seasons worth of such pairs was then ordered from smallest ETKF variance prediction to largest ETKF variance prediction. This ordered list was then separated into approximately equally populated bins. The mean of the squares of the normalized differences in the bin together with the mean of the normalized ETKF variance predictions in the bin were then computed and displayed in Fig. 8. (a) Unconditioned VR, Hybrid DA 6
4
2
8
Normalized Sample Signal Variance
Normalized Sample Signal Variance
8
(b) Unconditioned VR, Isotropic DA 6
4
2
0
0 0
2
4
6
8
10
12
0
14
(c) Conditioned VR, Hybrid DA
6
4
2
4
6
8
10
12
14
8
Normalized Sample Signal Variance
Normalized Sample Signal Variance
8
2
Normalized ETKF Signal Variance
Normalized ETKF Signal Variance
0
(d) Conditioned VR, Isotropic DA 6
4
2
0 0
2
4
6
8
10
12
14
Normalized ETKF Signal Variance
0
2
4
6
8
10
12
14
Normalized ETKF Signal Variance
Figure 8 Normalized sample signal variance versus ETKF normalized signal variance where in (a) and (c) hybrid DA was used while in (b) and (d) isotropic DA was used. In (a) and (b), unconditioned VR selection was used whereas in (c) and (d) conditioned VR selection was used. Units are non-dimensional. The solid and dashed lines connects data points based on bins of 1024 and 256, respectively.
This procedure is identical to that described in Majumdar et al., (2001, 2002) and Wang and Bishop (2003) except that their signals and signal variances are unnormalized whereas ours are normalized. Here, normalized variables are used to highlight the effect of differences in DA and VR selection on the ability of the ETKF to distinguish deviations of signal variance from climatological signal variance. In considering Fig. 8, recall that for very large sample sizes and accurate
15
variance prediction, the bin averaged normalized ETKF signal variance prediction ought to be equal to the sample variance of the normalized signal within each bin. The greater the difference between the signal sample variance of the bin with smallest ETKF signal variance and the signal sample variance of the bin of largest ETKF signal variance, the greater the utility of the signal variance prediction scheme. Comparison of Fig’s 8a and 8c with Fig’s 8b and 8d shows that the ETKF was better able to distinguish large signal sample variance from small signal sample variances with hybrid DA than with isotropic DA. Comparison of Fig’s 8a and 8b with Fig’s 8c and 8d shows that the ETKF distinguished a larger range of signal sample variances with conditioned VRs than with unconditioned VRs. One explanation for this second result is that with conditioned VRs, the upper bound of possible signals from targeted observations increases while the lower bound is largely unchanged. Another explanation is that ETKF signal variance predictions are more accurate with conditioned VRs. In considering Fig. 8, note that the oscillations of the solid and dashed curves are indicative of errors in our estimates of sample signal variance due to finite sample size. In particular, note that the dashed curve, which pertains to bins of 256, is much noisier than the solid curve which pertains to bins of 1024. With this reservation in mind, we note that with conditioned VRs and hybrid DA, Fig. 9c indicates that the ETKF can distinguish sample signal variances ranging from 0.063 to 7.5 (dashed line) and 0.12 to 4.7 (solid line) of the seasonally averaged signal variance due to targeted observations. With isotropic DA and with unconditioned VRs, the range of sample signal variances distinguished by the ETKF is less than half of this range being reduced to 0.39 to 3.4 (dashed line) and 0.37 to 2.8 for (solid line) of the seasonally averaged signal variance (Fig. 2b). This result highlights the sensitivity of the utility of targeted observations to the VR selection scheme and the DA scheme. Figure 8 summarizes the ability of the ETKF to predict the changes in the forecast in the VR due to targeted observations. One might assume that relatively large changes in the forecast should generally correspond to relatively large reductions in forecast error variance. In the next subsection, we examine the extent to which this assumption is justified for the DA schemes considered here. (c) ETKF predictions of reduction in forecast error variance Signal variance due to targeted observations is equal to the corresponding reduction of forecast error variance provided observation and forecast errors are normally distributed, the observation operator H is linear and the error covariances used for DA are accurate (see e.g. Bishop et al., 2001). Under these conditions, the DA scheme is optimal in the sense that analyses are maximal likelihood and minimum error variance state estimates (e.g. Daley, 1991). Hence, with optimal DA, signal variance prediction is equivalent to reduction in forecast error variance prediction; and in the limit of an infinite number of bins with an infinite number of realizations within each bin, the normalized signal variance versus ETKF signal variance plots shown in Fig. 8 would be identical to corresponding plots of Fig. 9 (see below) showing normalized reduction in forecast error variance versus normalized ETKF signal variance. To construct the panels in Fig. 9, the 250x64 (there are 64 grid points in each VR) data pairs are ordered and divided up into approximately equally populated bins in the same way as in Fig. 8. (Precisely equally populated bins are only possible when the number of bins is a factor of the number of data pairs). The reduction in forecast error variance due to targeted observations for each bin of trials is then estimated by taking the difference between the bin-averaged squared forecast error without and
16
with targeted observations3. The reduction in forecast error sample variance for each bin is then normalized in precisely the same way as the signal sample variances in Fig. 8 were normalized; i.e., each reduction in forecast error sample variance was divided by the climatologically averaged signal variance4. Also, the ETKF signal variance predictions were normalized in the same way as for Fig. 8. Finally, normalized reduction in forecast error variance for each bin is plotted against normalized ETKF signal variance prediction for each bin. (a) Unconditioned VR, Hybrid DA
6
4
2
8
Reduction in Normalized Forecast Error Variance
Reduction in Normalized Forecast Error Variance
8
(b) Unconditioned VR, Isotropic DA
6
4
2
0
0 0
2
4
6
8
10
12
0
14 8
6
4
2
Reduction in Normalized Forecast Error Variance
Reduction in Normalized Forecast Error Variance
8
(c) Conditioned VR, Hybrid DA
2
4
6
8
10
12
14
Normalized ETKF Signal Variance
Normalized ETKF Signal Variance
0
(d) Conditioned VR, Isotropic DA
6
4
2
0 0
2
4
6
8
10
12
14
Normalized ETKF Signal Variance
0
2
4
6
8
10
12
14
Normalized ETKF Signal Variance
Figure 9. As in Fig. 8, but here normalized reduction in forecast error variance versus ETKF normalized signal variance is plotted.
The solid lines (bins of 1026) in Fig. 9 are remarkably similar to the solid lines in Fig. 8. The dashed lines (bins of 256) in Fig. 9 are noisier than their counterparts in Fig. 8. An increase in noisiness is to be expected because the reduction in forecast error variance represents the difference between two sample variances whereas the sample signal variance is just a sample variance. Despite the noise, the dashed curves in Fig. 9 are still qualitatively similar to those in Fig. 8 for all points except for the points corresponding to the largest ETKF signal variances. The reductions in forecast error variance associated with these end points is smaller than the normalized sample signal variance of the corresponding points in Fig. 8 for both DA and VR selection schemes. However, note that there is no evidence of this trend in the end points of the solid lines corresponding to bins of 1024. With bins of 1024, the 3
Note that because verification observation error variances are the same for forecasts with and without targeted observations. The average difference between errors with and without targeted observations is approximately equal to the difference between verification innovations with and without targeted observations. Consequently, verifying observations may be used as a substitute for verifying truth in this approach. 4 Note that we could have chosen to normalize the reduction in forecast error sample variances by the average of all the reductions in forecast error variances. Graphs (not shown) for this normalization are qualitatively similar to Fig 10 in terms of the relative slopes and ranges of the lines in the four panels of Fig. 10.
17
end points of the reduction in normalized forecast error variance line for Fig’s 9a, 9b and 9d exceed the end points of the normalized sample signal variance lines in Fig’s 8a, 8b and 8d. Thus, there is no irrefutable evidence of an increase in sub-optimality with increasing signal variance even though one might expect sub-optimality to increase for extreme signal variances because when the first guess lies far from the truth (a) non-linear dynamics are more likely to lead to non-Gaussian error statistics, and (b) there is likely to be a high gradient coherent feature such as a front or vortex in the vicinity of the targeted observations for which the isotropic error covariances used by isotropic DA and hybrid DA are a poor approximation. Given the excessive noisiness of the dashed curves (bins of 256), we will restrict the remainder of our discussion to the solid lines (bins of 1024). As in Fig. 8, but to a lesser degree, comparison of Fig’s 9a and 9c with Fig’s 9b and 9d shows that the ETKF is better able to distinguish large reductions in forecast error variance from small reductions when hybrid DA is used than it is when isotropic DA is used. As in Fig. 8, but to a lesser degree, comparison of Fig’s 9a and 9b with Fig’s 9c and 9d shows that for both isotropic and hybrid DA, the ETKF distinguished a larger range of reductions in forecast error variance for conditioned VRs than for unconditioned VRs. 4. CONCLUSIONS For both hybrid and isotropic DA and for both 16 and 64 member ensembles, it was found that in the absence of adaptive observations, the largest forecast errors occurred in conditioned VRs, the 2nd largest in unconditioned VRs and 3rd largest in climatological VRs. For each of the three VRs, the ETKF was used to select two adaptive observations and the errors of forecasts that used these adaptive observations were compared against those that didn’t. Conditioned, unconditioned and climatological VRs respectively led to the greatest, 2nd greatest and 3rd greatest reductions in mean square VR forecast error due to adaptive observations. These reductions ranged from 33% for hybrid DA with conditioned VRs and a 64 member ensemble to 15% for isotropic DA with climatological VRs and a 16 member ensemble. The degree and statistical significance of the superiority of the error reductions in conditioned VRs over those in unconditioned VRs was smaller for isotropic DA than for hybrid DA. Empirical VRs were defined to be the VRs that had the largest errors without targeted observations. It was found that the targeted observation induced reduction in mean square forecast error for empirical VRs ranged from 12% for isotropic DA, climatological VRs and a 16 member ensemble to 22% for hybrid DA, conditioned VRs and a 64 member ensemble. Taking 2 targeted observations increased the total number of observations by 3% and decreased the domain wide mean square error by 4%-9%. Other tests showed that the ETKF’s ability to distinguish occasions where the forecast change due to targeted observations is likely to be large from occasions where the change is likely to be small was greater with Hybrid DA than with Isotropic DA. To a lesser degree, the ETKF’s ability to distinguish occasions where the forecast improvement due to targeted observations was large from occasions when it was small was also greater with hybrid DA than with isotropic DA. The errors with hybrid DA were smaller than those with isotropic DA. The hybrid DA approach is one of many possible pathways via which flow-dependent error variances and/or error covariances might be included in operational data assimilation schemes. Presumably, the benefits we found from hybrid DA would also be found in other approaches to flow-dependent error covariance modeling.
18
Data voids are more marked in the ocean than the atmosphere and it appears likely that a larger fraction of the oceanic observation network will be adaptive (Rudnick et al. 2004). Consequently, the work of Morss et al. (2001) suggests that the advantages of targeting observations using DA schemes with flow dependent covariances and conditioned VRs over isotropic DA and unconditioned VRs will be greater in oceanic applications than in atmospheric applications. Although the results for the 64 member ensemble experiments were all better than the 16 member ensemble results, the value of the larger ensemble was clearest in our finding that the targeted observation induced reduction in forecast error within empirical VRs (recall that these are the forecast bust regions) was insensitive to the type of VR selection method used when only 16 ETKF ensemble members were used. Under this measure, it was only with 64 ETKF ensemble members that the ensemble based conditioned and unconditioned VR selection methods gave greater empirical VR error reductions than the climatological VR selection method. This result highlights the need for further studies on the sensitivity of the effectiveness ensemble based VR selection methods to ensemble size in higher dimensional systems to that considered here. The proposed THORPEX Interactive Grand Global Ensemble (TIGGE) may help overcome the computational obstacles that currently thwart such research. ACKNOWLEDGMENTS The authors gratefully acknowledge National Science Foundation Grant ATM-98-14376. Craig Bishop received support under ONR Project Element 0601153N, Project Number BE-033-0345 and also ONR Grant number N00014-00-10106. Much of the revision for this paper occurred while Dr Bishop was visiting Meteo-Galicia and expresses gratitude for their help. REFERENCES Bachmayer, R. and Leonard, N.E., 2002: Vehicle Networks for Gradient Descent in a Sampled Environment. 41st IEEE Conference on Decision and Control, 2002 Baker, N.L and R. Daley, 2000: Observation and background adjoint sensitivity in the adaptive observation targeting problem. Quart. J. Roy. Meteor. Soc., 126, 1431-1454. Bergot, T., G. Hello, A. Joly, and S. Malardel, 1999: Adaptive observations: A feasibility study. Mon. Wea. Rev., 127, 743–765. Bergot, T. and A. Doerenbecher, 2002: A study on the optimization of the deployment of targeted observations using adjoint-based methods. Quart. J. Roy. Meteor. Soc., 128, 1689-1792 Bishop, C. H., B. J. Etherton and S. J. Majumdar, 2001: Adaptive sampling with the ensemble transform Kalman Filter Part I: Theoretical aspects. Mon. Wea. Rev., 129, 420-435. Buizza, R., and A. Montani, 1999: Targeted observations using singular vectors. J. Atmos. Sci., 56, 2965-2985.
19
Daley, R., 1991: Atmospheric Data Analysis. Cambridge University Press, 457 pp. Dee, D. P., 1995: On-line estimation of error covariance parameters for atmospheric data assimilation. Mon. Wea. Rev., 123, 1128-1145. Doerenbecher A., and T. Bergot, 2001: Sensitivity to observations applied to FASTEX cases. Nonlinear processes in geophysics, 8, (6), 467–481. Etherton, B. J., and C. H. Bishop, 2004: Resilience of hybrid ensemble/3D-Var analysis schemes to model error and ensemble covariance error. Mon. Wea. Rev., 132, 1065-1080. Gelaro, R., R. Langland, G. D. Rohaly, and T. E. Rosmond, 1999: An assessment of the singular vector approach to targeted observations using the FASTEX data set. Quart. J. Roy. Meteor. Soc., 125, 3299–3328. Gelaro, R., Rosmond, T., and R. Daley, 2002: Singular Vector Calculations with an Analysis Error Variance Metric. Mon. Wea. Rev., 130, 1166–1186. Hamill, T. M., and C. Snyder, 2000: A hybrid ensemble Kalman filter/3D-variational analysis scheme. Mon. Wea. Rev., 128, 2905-2919. Hamill, T. M., and C. Snyder, 2002: Using Improved Background-Error Covariances from an Ensemble Kalman Filter for Adaptive Observations. Mon. Wea. Rev., 130, 1552–1572. Joly, A., K. A. Browning; P. Bessemoulin; J.-P. Cammas; G. Caniaux; J.-P. Chalon; S. A. Clough; R. Dirks; K. A. Emanuel; L. Eymard; R. Gall; T. D. Hewson; P. H. Hildebrand; D. Jorgensen; F. Lalaurette; R. H. Langland; Y. Lemaitre; P. Mascart; J. A. Moore; P.O. G. Persson; F. Roux; M. A. Shapiro; C. Snyder; Z. Toth; R. M. Wakimoto, 1999: Overview of the field phase of the Fronts and Atlantic Storm-Track EXperiment (FASTEX) project. Quart. J. Roy. Meteor. Soc., 125, 3131-3164. Langland, R. H., Z.Toth, R.Gelaro, I.Szunyogh, M.A.Shapiro, S.J.Majumdar, R.E.Morss, G.D.Rohaly, C.Velden, N.Bond and C.H.Bishop, 1999: The North Pacific Experiment (NORPEX-98) Targeted observations for improved weather forecasts. Bull. Amer. Meteor. Soc., 80, 1363-1384. Lermusiaux, P.F.J., 2001. Evolving the subspace of the three-dimensional multiscale ocean variability: Massachusetts Bay (.pdf). J. Marine Systems, Special issue on ``Three-dimensional ocean circulation: Lagrangian measurements and diagnostic analyses'', (29), 1-4, 385-422. Leutbecher, M., 2003: A reduced-rank estimate of forecast error variance changes due to intermittent modifications of the observing network. J. Atmos. Sci., 60, 729742. Lorenz, E.N. and K.A. Emanuel, 1998: Optimal sites for supplementary observation sites: Simulation with a small model. J. Atmos. Sci., 58, 210-232.
20
Mansfield, D., Richardson, D. , Truscott, B., 2005: An Overview of the Atlantic THORPEX Regional Campaign (A-TRec). World Meteorological Organization Proceedings of the 1st THORPEX symposium. Majumdar, S. J., C. H. Bishop, B. J. Etherton, I. Szunyogh, and Z. Toth, 2001: Can an Ensemble Transform Kalman Filter predict the reduction in forecast error variance produced by targeted observations? Quart. J. Roy. Meteor. Soc., 127, 2803-2820. Majumdar, S. J., C. H. Bishop, B. J. Etherton, and Z. Toth, 2002: Adaptive sampling with the ensemble transform Kalman Filter Part II: Field program implementation. Mon. Wea. Rev., 130, 1356-1369. Molteni, F., R. Buizza, T. N. Palmer, and T. Petroliagis, 1996: The ECMWF ensemble prediction system: Methodology and validation. Quart. J. Roy. Meteor. Soc., 122, 73119. Morss, R. E., Emanuel, K. A., Snyder, C., 2001: Idealized Adaptive Observation Strategies for Improving Numerical Weather Prediction. J. Atmos. Sci., 58: 210-232 Palmer, T. N., R. Gelaro, J. Barkmeijer, and R. Buizza, 1998: Singular vectors, metrics, and adaptive observations. J. Atmos. Sci., 55, 633-653. Parrish, D. F., and J. C. Derber, 1992: The National Meteorological Center's Spectral Statistical Interpolation Analysis System. Mon. Wea. Rev., 120, 1747-1763. Pu, Z., and E. Kalnay, 1999: Targeting observations with the quasi-linear inverse and adjoint NCEP global models: Performance during FASTEX. Quart. J. Roy. Meteor. Soc., 125, 3329-3338. Rudnick, D. L., R. E. Davis, C. C. Eriksen, D. M. Fratantoni, and M. J. Perry, 2004, Underwater Gliders for Ocean Research. Journal of the Marine Technology Society. 38, 48-59. Szunyogh, I., Z. Toth, R. E. Morss, S. J. Majumdar, B. J. Etherton, and C. H. Bishop, 2000: The effect of targeted dropsonde observations during the 1999 Winter Storm Reconnaissance Program. Mon. Wea. Rev., 128, 3520-3537. Szunyogh, I., Z. Toth, A. V. Zimin, S. J. Majumdar, and A. Persson, 2002: Propagation of the Effect of Targeted Observations: The 2000 Winter Storm Reconnaissance Program. Mon. Wea. Rev., 130, 1144–1165. Toth, Z., I. Szunyogh, C. H. Bishop, S.J. Majumdar, R. Morss, J. Moskaitis, Reynolds, D., Weinbrenner, D., Michaud, D., Surgi, N., Ralph, M., Parrish, J., Talbot, J., Pavone, J. and S. Lord, 2002: “Adaptive observations at NCEP: Past, present and future”. Preprints for the Symposium on Observations, Data Assimilation, and Probabilistic Prediction, 13-17 January 2002, Orlando, FL, 185-190. Toth, Z., I. Szunyogh, C. H. Bishop, S.J. Majumdar, R. Morss and S. J. Lord, 2001: “On the use of targeted observations in operational numerical weather prediction”. Pp. 72-79 in Preprints of the 5th AMS symposium on integrated observing systems, 15-19
21
January 2001, Albuquerque, NM, USA. American Meteorological Society. Toth, Z., I. Szunyogh, C. H. Bishop, R. E. Morss, B. J. Etherton, and S. J. Lord, 1999: “The 1999 Winter Storm Reconnaissance Program”. Pp. 27-32 in Preprints of the 13th AMS conference on numerical weather prediction, 13-17 September 1999, Denver, CO, USA. American Meteorological Society. Wang, X. and C.H. Bishop, 2003: A comparison of Breeding and Ensemble Transform Kalman Filter Forecast Schemes. J. Atmos. Sci., 60, 1140-1158. Wang, X., C.H. Bishop and S.J. Julier, 2004: Which is better, an ensemble of positive/negative pairs or a centered spherical simplex ensemble? Mon. Wea. Rev. 132, 1590-1605.
22
