182
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
A Multiple Scale State-Space Model for Characterizing Subgrid Scale Variability of Near-Surface Soil Moisture Praveen Kumar
Abstract—This paper addresses the problem of characterizing variability of soil moisture at various scales by combining information, such as measurements and soil hydrologic properties, available at different scales. This problem is motivated by the need to provide a way to predict subgrid/subpixel variability from measurements made at satellite footprint scale. A meandifferenced multiple scale fractal model is developed for soil moisture. The salient features of this model are as follows. 1) Differences in soil moisture in various hydrologic groups are modeled through a difference in mean, while the fluctuations are assumed independent of the mean. 2) Mean soil moisture is linearly related to available water capacity of the soil. 3) Fluctuations are modeled as fractional Gaussian noise. Estimation techniques based on multiresolution trees are implemented to obtain the values at multiple scales. Since estimation is a smoothing process that may not provide a good representation of the variability, particularly in regions where there are no observations, a complementary conditional simulation technique is developed. This allows us to construct synthetic fields that are representative of the intrinsic variability of the process. The technique is applied to problems of estimation and conditional simulation for the following scenarios: domain with missing values, sparsely sampled data, in domain outside of where measurements are available, and at scales smaller than at which measurements are available. Index Terms— Conditional simulation, Kalman filter, multiscale model, optimal estimation, remote sensing, soil moisture.
I. INTRODUCTION
S
OIL MOISTURE has a significant impact on ecosystem, hydrologic, and atmospheric dynamics. It influences the ecosystem processes at several time scales through its control on nutrient availability, development of plant canopies, and their physiological activities. Dry soil conditions limit water and nutrients to bio-organizms living in and on the soil, resulting in slow biogrowth. Under moist soil conditions, water and nutrients are more readily available to bio-organizms and the soil is more productive in terms of supporting plant and animal life. Along with the vegetation it sustains, soil moisture controls the partitioning of available solar energy at the ground surface into sensible and latent heat fluxes, thereby providing Manuscript received May 27, 1997; revised January 26, 1998. This work was supported by a University of Illinois Research Board grant and NASA Grants NASA-NAGW-5247 and NASA-NAG5-3661. The author is with the Hydrosystems Laboratory, Department of Civil Engineering, University of Illinois, Urbana, IL 61801 USA (e-mail:
[email protected]). Publisher Item Identifier S 0196-2892(99)00031-5.
a link between the water and energy balance at the surface of the earth as well as regulating the surface temperature. The latent heat flux feeds into the atmospheric circulation and is known to influence formation of clouds and precipitation at short time scales under certain atmospheric conditions. At longer time scales, land-surface soil-moisture variability can produce significant changes in atmospheric circulation. It also determines the fraction of precipitation partitioned into infiltration and surface runoff components. Given the crucial role of soil moisture in various processes, it is important to estimate its variability over large space and time scales. However, it is known to have a highly heterogeneous distribution due to the vastly varying properties of topography, soil, vegetation, etc. Consequently, its measurement is a difficult task, particularly for large spatial extents and time periods. Recognizing the necessity of estimating soil moisture over large spatial scales, field campaigns, such as the Washita 1992 experiment, were conducted to assess the feasibility of soil-moisture measurement using passive microwave remote-sensing instruments [11]. Measurements 200-m resolution from aircraft-based electronat 200 ically scanned thinned array radiometer (ESTAR) passive microwave radiometer in a 610-km Little Washita watershed in Oklahoma showed that the soil texture is a very strong determinant of near-surface soil moisture [11]. The stochastic characteristics of the soil-moisture variability showed a long range dependence akin to fractional Brownian random fields [23]. These two results indicate the feasibility of predicting and/or estimating the subgrid/subpixel variability of soil moisture by combining the measurements at large scales (low resolutions), which would be available from space-based remote-sensing technology in the near future, along with an existing database of soil properties, such as the STATSGO [25]. These predictions and/or estimations can be further improved through in situ measurements wherever available, such as the Illinois soil-moisture network [12]. Typically, such multiple scale measurements involving remote sensing and in situ observation have the characteristic that fine scale measurements of limited coverage are embedded within coarse scale measurements of larger coverage (see Fig. 1). Further analysis is usually performed to integrate the information across scales. This paper addresses the problem of characterizing variability of soil moisture at various scales by combining information, such as its measurements available at different scales and
0196–2892/99$10.00 1999 IEEE
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
Fig. 1. Schematic showing an idealized embedding of fine scale measurement grid of limited coverage within a coarse scale grid of larger coverage for a two-level scheme.
soil hydrologic properties. This problem is motivated by the need to provide a way to predict subgrid/subpixel variability. In particular, we address three distinct but related issues, as follows. 1) Given measurements at two or more scales (for example, using remote-sensing and in situ techniques), how can we obtain optimal consistent estimates across scales? This problem has been addressed through the development of the multiple scale Kalman filtering algorithm [1]–[3], [5]–[7]. The key to this algorithm is the treatment of the scale parameter akin to time, such that the description at a particular scale captures the features of the process up to that scale that are relevant for the prediction of finer scale features. The Kalman filtering technique is used to obtain optimal estimates of the states described by the multiple scale model using observations at a hierarchy of scales [6], [7]. We discuss the usefulness of this technique for the estimation of soil moisture from measurements at multiple scales using a fractal prior model [23]. 2) Estimation techniques that are based on minimizing a cost function, typically involving the variance, underestimate the variability, and the resulting estimated field is smoother than the true field. Therefore, conditional simulation [14], rather than estimation, is used to characterize the variability of the underlying field. The random fields obtained by this technique preserve the observed values in which they are known and at the same time simulate the intrinsic fluctuations at other locations that are consistent with the mean and the covariance of the process. Given the wide usefulness of conditional simulation techniques for the usual one- and two-dimensional (1-D and 2-D) processes, an algorithm for the multiple scale framework is developed. This technique provides equiprobable random fields at multiple scales that have the best possible representation at scales and locations where observations are made, and they preserve the variability and neighborhood dependence. The conditionally simulated fields can then be used for several applications, such as assessment of subgrid variability, inputs
183
to physical models or design of sampling strategies, etc. These are of significant importance for soil-moisture fields as they show significant variability even at very small scales. We apply the multiple scale conditional simulation technique to characterize the variability of soil-moisture fields in regions where no measurements are directly available. 3) Given measurements at some large scale, how do we obtain simulated fields at smaller scales that are consistent with the measurements at the larger scale? This may be considered as a scale extrapolation problem. The motivation for this problem is to provide a methodology to infer subpixel variability from satellite-based instruments that will provide measurements at a resolution of a few kilometers. A state-space model relating the soil moisture with the underlying soil hydrologic properties is developed for obtaining conditionally simulated random fields at scales smaller than those at which measurements are available. The soil-moisture state is modeled as a mean-differenced fractional Brownian or fractal process. The methodology is general and can be used to address similar problems in a wide variety of fields. This paper is organized as follows. Optimal estimation using the multiple scale Kalman filtering [6] is briefly discussed in Section II. The details of the algorithm are described in the Appendix. The method of multiple scale conditional simulation is developed in Section III. Extensive statistical analysis of soil-moisture measurements obtained using ESTAR instrument during the Washita 1992 field campaign is performed in Section IV to establish the state-space soil-moisture model. The estimation and simulation problem using the soilmoisture field is described in Section V. Concluding remarks are delegated to Section VI. II. REVIEW
OF
MULTIPLE SCALE MODELING
A. Multiple Scale State-Space Model The following description briefly explains the algorithm of multiple scale Kalman filtering [1]–[3], [5]–[7]. Please consult the original work for details. Consider the problem of disaggregating a 2-D random field from coarse to fine resolution. At the coarsest resolution, the field will be represented by a single value (see Fig. 2). At the next resolution, there will be values, and, in general, at the th resolution, we obtain values. The values of the random field can be described , where represents the resolution on the index set the location index. The scale-to-scale decomposition and can be schematically depicted as a tree structure (quadtree for 2-D processes), as shown in Fig. 2. To describe the model, let us use an abstract index to specify the nodes on the tree specify the parent node of (see Fig. 2). Then the and let multiple scale stochastic process can be represented as (1) Here
is the state vector and
and
are
184
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
distributed with known covariance matrices, i.e., (2) (3) is the identity matrix and and are where independent. It is further imposed that the model is Markovian when going from the coarse to fine scale, i.e., conditioned on any node of the tree, each of the subtrees connected to this node are independent. For example, conditioned on the top in Fig. 2, each of the distinct subtrees below this node node are independent. For this to hold, it is only required be independent from scale to scale. Interpreting the that states at a given level of the tree as a representation of the process at one scale, we see that (1) describes the evolution of the process from coarse to fine scale. The term represents the interpolation or prediction down to the next finer represents new information added as the level, and process evolves from one scale to the next [5]. It is easy to of the verify that the covariance state at node evolves as (4) Notice that the form of (1) and (4) do not change whether we deal with 1-D or multidimensional processes. Following Verghese and Kailath [28], fine-to-coarse scale evolution can be written as [6], [7]
(a)
(5) where (6) is uncorrelated along all upward paths of the tree and and has a variance given by E (7) Multiple scale stochastic models can be used to make estimates of processes from noisy measurements obtained at different resolutions. If we are given noisy measurements of the state , we can develop an estimation problem using the combination of (1) and the following equation: (8) (b) Fig. 2. (a) Structure of a multiple scale random field is shown. The values at various grid locations (i; j ) are given as x (i; j ), where m is the resolution index. At the coarsest resolution (m = 0), the field is represented by a single state vector, and generally at the mth resolution, there are 4 state vectors. (b) Abstract representation of the multiple scale decomposition. The abstract index refers to a node in the tree, and refers to the parent node.
m
m
matrices of appropriate sizes (see the Appendix). The root node , which represents the properties of the mean state over are normally the entire domain, and the driving noise
characterizes the measurement variance. It is indewhere and for all and . pendent of can specify, in a very general way, measureThe matrix ments taken at different spatial locations and at different scales. specifies the covariance of the measurement errors . Equations (1) and (8) can be solved jointly to obtain estimates , hereafter denoted as (see the Appendix). Notice of that this is a very attractive technique as it enables us to combine estimation and filtering while exploiting hierarchical measurements at different scales.
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
Fig. 3. Schematic of two-pass estimation in the multiple scale framework. First, the upward pass propagates information up the tree. At each tree node the estimate incorporates all measurements on that node and its descendants. Then, during the downward pass, the information is propagated down the tree. The estimates at each tree node are now based on information on all nodes on the tree.
B. Optimal Estimation and Smoothing Keeping in mind that the multiscale tree model is Markovian, for any node , the processing of the data in the subtree beneath it can be accomplished through independent processing of the data in each of the descendent subtrees. The based on the basic scheme for optimal estimation of proceeds in two steps of an upward and measurements downward sweep. This is explained below. 1) Upward or Fine-to-Coarse-Scale Sweep: The first step of the estimation applies Kalman filter prediction going from fine to coarse scale using (5) (see Fig. 3). Therefore, for each coarse scale node, there will be predicted values, one from each of the child nodes. These predictions are then merged to obtain a single predicted value. This merge step does not have any counterparts in the usual time-domain Kalman filtering schemes. Using this merged prediction, the update step that incorporates the observations (8) is the usual Kalman filter step involving the Kalman gain matrix. 2) Downward or Coarse-to-Fine-Scale Sweep: After the above procedure is carried out from the finest to the coarsest scale, the downward sweep generalizes the Rauch–Tung–Stribel algorithm [22] and produces smoothed best estimates and error variances on every node based on all data. Thus, calculations on every node are performed once on each of the upward and downward sweeps. The rationale for using the upward sweep first is that this step carries information from the fine to the coarse scale, which, during the downward sweep, is passed onto other nodes that are not under the same to contribute to subtree. For example, in Fig. 2, for (or its descendants), we need to first move up to and then down the tree. The details of these computations are summarized in the Appendix. Here it suffices to make the following observations. • Missing observations at any node are handled easily by setting the updated estimates to be the same as the predicted value. • Estimation errors are computed at each node both during the upward and downward pass. • Multiple scale evolution (1) easily incorporates multiple variables that might be covarying or related through a governing equation. • Equation (1) is quite general and can be used to model a wide range of processes, such as Markov random fields
185
[17] and fractional Brownian motion (fBm) or fractal fields [6], [9]. In this paper, we only focus on the latter, which provides a rich class of models for a wide range of hydrologic random fields, such as log hydraulic conductivity and soil moisture. This is discussed in the following subsection. • Heterogeneities are handled easily by specifying parameters that vary from one node to another. We will take advantage of this in developing the conditional simulation model. • It is important to reiterate that this method considers random field frozen in time and no temporal evolution is involved. The model considers only variability across scales. C. Fractal Prior Model Equation (1) provides a very general modeling framework. As mentioned before, it is possible to choose and such that (1) leads to a fractal model. Consider the case in which the state is a scalar and the model depends only on the of node [ for the root node] resolution index and not on locations and . Then a canonical fractal model can be written as (9) is a constant that characterizes the total variance of where the process and depends on the fractal dimension or spectral decay rate [5], [9], [17]. For a 2-D process with spectrum , is given by (10) corresponds to Brownian field with For example, . The validity of this result rests Hurst exponent on the wavelet theory, in which it is shown that the wavelet coefficients of a fractional Brownian process are approximately uncorrelated and the variance decays as a power law function of scale [30]. III. MULTIPLE SCALE CONDITIONAL SIMULATION A. Theoretical Foundation allows several reThe multiple scale random field alizations based on the assumed model structure. The true is one of such realizations that is commensurate field with the modeling objective, such as preservation of mean and neighborhood dependence (the covariance in a 2-D random are available for selected field). When measurements , , the objective of the estimation algorithm for all using the obis to obtain an estimated field servations and incorporate their measurement uncertainty and the neighborhood dependence. The neighborhood dependence can take into account the spatial proximity when considering a fixed scale, or the proximity in scale when considering a fixed location, or both. Our objective in multiple scale simulation is that exhibit similar to generate equiprobable fields neighborhood dependence in space and scale. The real and
186
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
simulated realizations and , respectively, of differ from each other at given locations but preserve the neighborhood dependence. From the infinite realizations of , the objective of conditional simulation is to choose whose values at the measurement locations are that field as close to the estimated values as possible. The conditionally simulated and real fields have the same cluster of large and small values. The essential difference between the estimated field and a conditionally simulated field lies in the representation of the neighborhood variability. At sampling points that are closely located, both in space or scale, the estimated field provides a good representation of the variability. However, wherever measurements are sparse, the estimated field is smooth with loss of information on attributes such as the variability, connectivity, and distribution of extreme values [15], [16]. The conditionally simulated field does not suffer from this limitation. The tradeoff, however, is that the simulated values are not the best possible estimators of the true field. In fact, the variance of the simulated values is larger than that of the estimated values. Let us first consider the standard approach used in geostatistics [8] for the generation of a conditionally simulated , where denotes 2-D random field. The true field locations in a 2-D space, can be broken into the optimally (using kriging [16]) and an orthogonal error estimated , i.e., term
The modeling of the error process is done implicitly in the procedure described above using kriging for 2-D random fields. For our case of multiple scale processes characterized through a tree structure, we can adopt the same idea provided we can generate the error term through a model that preserves the same neighborhood dependence as . Recall that the multiple scale Kalman filtering proceeds first as an upward sweep in which fine scale measurement information is propagated to the root node. Then the downward sweep produces smoothed estimates that incorporate the denote information from the entire data set. Let the optimal estimate at node conditioned on measurements on node and its descendants during the upward pass, and denote the corresponding error variance. Let let and denote the optimal smoothed estimates during the downward pass and the corresponding error variances, respectively. Define the estimation error as (13) Therefore, by definition (14) The error variance obeys [6], [7] (see the Appendix) (15) where
(11)
(16)
is estimated and therefore known for all , Since by a field that it suffices to substitute is isomorphic to it; that is, it has the same covariance. This is easily accomplished by kriging a simulated field with values retained only at spatial locations where data are available and then taking the difference between the kriged field and the simulated field, i.e., by obtaining . The error term is and independent of . isomorphic to can be generated The unconditionally simulated field through any number of techniques, such as the turning bands method [27]. We can, therefore, generate our conditionally simulated field as
Luettgen and Willsky [19] have shown that a model for that reproduces (15) can indeed be developed. In fact, the model can be cast as a multiple scale model of the form given in (1). Without going through the technicalities of the derivation, consider the model
(12)
The above can be alternately written as a sum of two positive definite matrices [32]
This field is now known for all locations . Every new gives a new instance of the simulation field for the same estimated field , thereby giving several equiprobable fields. We can think of the above procedure more abstractly. is the projection of the Whereas the estimated field onto the space spanned by information available process from the measurements, the error term provides a modeled projection of onto the corresponding null space. The conditional simulation therefore adds the unknown null space information by generating it through a model to the range space. Hence, the key for conditional simulation is to find a model for the error process.
(17) denotes the simulated error process and where is a white noise process independent of . Using (4), it satisfies (15) provided is easy to verify that (18)
(19) is a positive definite matrix [see (43)]. can where then be obtained using the Cholesky decomposition of the right-hand side. The conditionally simulated field is simply (20) or (21) Equation (21) will henceforth be referred to as the multiple scale conditional simulation model. By recognizing that
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
when no measurements are available and , it is easy to verify that and and the model of the error is exactly . This is intuitive since, whenever the same as that of there is a lack of measurements, the unconditional model simulation provides the best characterization of the process. will be large in We should expect that the error term magnitude for nodes with no measurements. This is because the estimation errors for these nodes will be large. Consefield will be nonhomogeneous even if quently, the is homogeneous but sparsely sampled. The multiple scale model (17) enables us to easily incorporate this by specifying for each , and it can be computed. a different B. Application to Fractional Brownian Random Fields In order to assess the applicability of the estimation and simulation techniques discussed above, we apply them to a simulated fractional Brownian field using the model (9). Fractional Brownian models have found applications in a variety of physical processes. For example, log conductivity fields are often described by such models. Due to their generality, the results presented in this section are applicable to all such scenarios in which fractional Brownian models are useful. We first discuss some parameter estimation issues for the model (9). We then demonstrate the estimation and conditional simulation results using synthetic fractional Brownian fields. In particular, we discuss two cases involving noise-corrupted and sparsely sampled field. The applicability of this model for the study of the soil-moisture fields is discussed in the sections that follow. Parameter estimation for the model (9) can be obtained using either a maximum-likelihood technique or the method of moments. The maximum-likelihood technique for the multiscale model was developed by Luettgen and Willsky [18]. It maximizes the likelihood function obtained from a set of noisy measurements, assuming that the data correspond to the particular multiscale model. For the fractal model (9), and , where we need to estimate the two parameters it is found that the method gives a set of combinations that approximately have the same large likelihood [9]. Also, the estimation becomes computationally intensive, with an increasing number of scales, as required in this study. For this work, we use the simpler method of moments. Using (4) for the model (9), we can write (22) of the process at scale If we know the variance , and ignoring , which is the prior uncertainty in the estimation of the global mean state or the root node, we see that (23) The parameter , which completely determines , can be estimated either through the decay rate of the spectrum or through the slope of the log–log plot of variance at different
187
scales versus scale [31]. We use the latter, i.e., if constant
(24)
. For a 2-D process, is related to the fractal then . Once the estimated values and, dimension as can be obtained using (23). consequently, is determined, for all . For the measurement model (8), we use can be either specified as an arbitrary The parameter large value or evaluated as the uncertainty in the estimation of the global mean state. The usual estimator for the mean , where and has an uncertainty given as are the variances of the process and the measurement errors, . respectively. We use this value to specify the parameter 64 grid An isotropic fractional Brownian field on a 64 and variance with fractal dimension was generated using a spectral technique [29]. The estimated , , parameters for the generated field are . The synthetic field is shown in Fig. 4(a). and This field was corrupted with white noise with a signal-to] of zero [see Fig. 4(b)]. The noise ratio [ for all measurement noise was specified as nodes at the finest scale. Fig. 4(c) and (d) show the estimated field using the multiscale model and a conditionally simulated field, respectively. The standard deviation of the estimation error was uniformly 0.65 at the finest scale. Notice the smooth nature of the estimated field and loss of variability. Its resemblance to the original field is remarkable, demonstrating the ability of the technique to extract signal from noise. The conditionally simulated field, however, is more variable, reflecting the uncertainty due to noise corruption. Fig. 5(a) shows a randomly subsampled field where only 10% of the original points of data shown in Fig. 4(a) are retained. Fig. 5(b) and (c) show the estimated field and the estimation error, respectively. The estimated field remarkably captures the large-scale variability of the original field [compare with Fig. 4(a)]. Notice that the estimation error is nonhomogeneous with large values at locations with no meaused for conditional simulation surements. The parameter is a function of these errors, which is now different for each point. This provides an example in which the model is useful for fields whose parameters vary at different locations at a particular scale. A conditionally simulated field is shown in Fig. 5(d). Notice the large variability in regions with missing observations captured by this field. IV. MODEL FORMULATION A. Analysis of ESTAR Soil-Moisture Measurements In order to apply the multiple scale estimation and simulation technique to soil-moisture fields, we need to establish an appropriate model. We analyze the data collected during the Washita 1992 field experiment [11], obtained using the ESTAR passive microwave radiometer as brightness temperatures, and converted to volumetric soil-moisture content [11]. This experiment was a joint effort between NASA, USDA (United States Department of Agriculture), and several other government agencies and universities. Data were collected from
188
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
(a)
(b)
(c)
(d)
Fig. 4. Estimation from noise-corrupted fractional Brownian field: (a) original field on a 64 ratio of zero; (c) estimated field; and (d) simulated field.
2 64 grid; (b) noise-corrupted field with signal-to-noise
(a)
(b)
(c)
(d)
Fig. 5. Modeling of (a) subsampled fractional Brownian field shown in Fig. 4(a); (b) estimated field; (c) estimation error; and (d) simulated field.
June 10–18, 1992, except for June 15, 1992, in the 610-km Little Washita basin in Oklahoma. The data sets correspond to daily measurements of soil moisture in the top 5-cm layer. There were heavy rains just before and no rain during the experiment, giving a dry down scenario. There were several in situ measurements as well to provide the ground truth. The ESTAR microwave data are available on a grid of 93 rows
228 columns, with each grid box corresponding to a 200 200-m area, as shown in Fig. 6. The ground samples provide an average estimate over a 30 30-m area and are available at 24 locations within the ESTAR image domain. These were obtained as averages of nine samples on a square grid basis located 10 m apart [13]. In this section, using data analysis, we 1) establish the nature of dependence between soil-moisture
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
Fig. 6. Washita 1992 0–5-cm volumetric soil moisture percentage estimated with ESTAR for June 10–18, 1992. The data are on a 93 each pixel having a resolution of 200 200 m. Missing values are indicated as white areas within the domain.
2
and soil-hydrologic properties and 2) hypothesize a model for this dependence and establish its validity. This will then be used for multiple scale modeling. Fig. 7 shows the soil texture and vegetation categories at each of the grid points. It is quite evident from Figs. 6 and 7 that soil texture is a strong determinant of the surface soil moisture. In order to establish the nature of this dependence mathematically, we introduce some terminology. For a soil denote the porosity defined as the volume of matrix, let void space per unit volume of soil matrix. Then (25) where and are volumes of gas and water per unit volume of soil matrix, respectively. is also called the volumetric water content. The soil is considered saturated when all void space is filled with water. The saturated water content is equal to the porosity . The water held in the soil matrix can be used by the plants. During and immediately after a rainfall event, the void space gets filled and the water moves under the influence of gravity. This results in a reduction in soil-moisture content below the saturation level. “Field capacity” ( ) is defined as the water content at which initial rapid drainage due to gravity becomes negligible and is often measured as the water content at 33-kPa pressure [20, ch. 5]. As water content is further depleted due to evaporation or transpiration, it reaches a level at which plants can no longer extract water from the pore space due to the strong adhesive forces between water molecules and soil particle surface. This is called the “wilting point” ( ) and is usually measured as the water content at 1500–kPa pressure [20]. Water available to plants is generally considered to be that between field capacity and wilting point. This is called the “available water capacity,” which can be expressed
189
2 228 grid, with
in terms of percentage by volume simply as (26) Fig. 8(a) and (b) show the mean soil moisture for different soil textures and land use categories for the June 10 and 18 observations. The two dates are chosen for illustration to contrast the saturated and dry conditions. It is evident that the variability due to the soil texture is far greater than due to land use. Although the mean values reduce significantly due to dry down after eight days, the dependence on the soil texture is still dominant. Henceforth, the dependence on the land use will not be considered. Fig. 8(c) and (d) shows the box plot of the soil-moisture values for various texture classes for June 10 and 18. The box indicates the interquartile range. The middle white strip indicates the median. The whiskers are plotted at 1.5 the interquartile range, and the extreme values (values lying outside 1.5 the interquartile range) are individually marked. The width of the box is proportional to the number of values in each texture class. As shown in the figures, the moisture distribution for different texture classes are distinguished by the differences in the central values (i.e., the median), although some texture classes can be grouped together to reduce their number. Further discussion will show that this is unnecessary for our purpose. There seems to be no significant difference in the distribution within each texture class. B. Soil-Moisture Dependence on Soil Hydrologic Properties In order to model the differences in soil-moisture distribution for each soil texture class, as indicated in the above discussion, identify each texture class as a hydrologic group , , where is the total number of hydrologic groups in the domain of interest. We further define as the
190
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
(a)
(b)
(c)
Fig. 7. (a) Soil texture and (b) land use categories at the ESTAR measurement site. The soil texture classes are: 0—missing, 1—sand, 2—loamy fine sand, 3—fine sandy loam, 4—loam, 5—silt loam, 6—silty clay loam/clay loam, 7—pits, quarries, urban, 8—gypsum, and 9—water. The land use classes are: 0—missing, 1—range land, 2—crop land, 3—pasture, 4—forest, and 5—all others. (c) Available water capacity (percentage by volume) obtained from the STATSGO database.
mean soil moisture for the entire domain, henceforth called as the mean soil moisture for the the domain mean, and hydrologic group . We assume that the soil moisture at any consists of two components, given as location
soil moisture at
mean of hydrologic group at (27) fluctuations
The above can be equivalently written as
(28)
with the condition that . gives the deviation of the soil-moisture mean of the hydrologic group from the are random fluctuations, domain mean. We assume that normally distributed with mean equal to zero and variance , i.e., the behavior of the fluctuation is independent of the hydrologic group. In order to further account for the long range dependence akin to fractional Brownian motion observed in soil-moisture fields [23], we model as a fractional Gaussian . noise [21], i.e., has a spectrum Equation (28) can be tested using the usual analysis of variance techniques [10]. The null hypothesis to be tested for all . Using the F distribution test, it is was found that this hypothesis can be rejected, for all of the days of the measurement, for p-values that are almost negligible, indicating that there is a considerable difference in
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
(a)
(b)
(c)
(d)
Fig. 8. Demonstrating the dependence of soil moisture on various soil texture and land use categories displayed in Fig. 7 for (a) June 10, 1992, and (b) June 18, 1992. The domain mean is indicated by the long horizontal line, whereas the mean for each individual category is plotted as short horizontal lines. The category indexes overlap when their mean are not very different. (c). (d) Boxplot comparison of soil moisture for various soil texture classes for June 10 and 18, respectively.
Fig. 9. Showing the distribution of the residual 0 (i; j ) for the June 10 and 18, 1992, data sets.
the mean values of soil moisture belonging to each hydrologic group. Fig. 9 shows the distribution of the residuals for June 10 and 18, 1992, data sets. The residuals show a near-normal distribution. Some deviations from normality are observed and found to be greater for the wet days than for the dry days. Some of this deviation can be attributed to the presence of missing values in the domain. Although model (28) with modeled as fractional Gaussian noise is sufficient to characterize the differences in soilmoisture distribution for different hydrologic groups, while at the same time accounting for their long range dependence, it does not include the dependence on soil-hydrologic properties. In order to accomplish this, we assume that for each of
191
Fig. 10. Showing the dependence of mean soil moisture for each hydrologic group on the available water capacity for different hydrologic groups for the June 10 and 18, 1992, data sets.
the hydrologic groups, the mean soil-moisture state depends linearly on a hydrologic parameter . The parameter discriminates between the water-holding characteristics of different hydrologic groups . For it to be useful over large aerial extents for satellite-based measurements, it should be easily measurable or available through databases, such as the is equal to the mean STATSGO [25]. We assume that the available water capacity AWC for the hydrologic group . In could areas where topographic controls are predominant, be related to the mean of the topographic index [4]. This is not pursued here, as there is no evidence that topographic controls on surface soil moisture are significant for the Little Washita basin. Available water capacity is a standard parameter of the STATSGO database, with aerial extent covering the continental United States. AWC obtained from these data will be assumed to represent the mean for the hydrologic group. We will also assume that the distinct values of AWC define the extent of a soil hydrologic group. Fig. 7 (bottom) shows the AWC obtained from the STATSGO database for the domain on which soil-moisture measurements are available. By comparing this with Figs. 6 and 7 (top), it is evident that this parameter can discriminate the different hydrologic groups. Henceforth, these data will be used for our study. and With the assumption of linear dependence between (or ), we have AWC
(29)
Here, AWC characterizes the deviations of the AWC of group from that of the domain mean AWC. The parameter is a “wetness index” that characterizes the degree of wetness of the soil (in relation to the AWC) and will be large when soil is saturated and small when we have dry conditions. Fig. 10 versus AWC for June 10 and 18 data shows the plots of sets using least-trimmed-squares regression [24]. The wetness index , which corresponds to the slope, is 2.08 and 1.33 for June 10 and 18, 1992, data sets, respectively. Goodness of fit for the other dates were comparable.
192
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
V. APPLICATIONS
With (29), our model (28) reduces to AWC
(30)
is fractional Gaussian noise. In summary, the where salient features and key assumptions in the model are as follows. 1) Differences in soil moisture in various hydrologic groups are modeled only through a difference in mean, while the fluctuations are assumed independent of the mean. 2) Mean soil moisture is linearly related to available water capacity of the soil. 3) Fluctuations (residuals) show stochastic long range dependence that are modeled as fractional Gaussian noise. C. Multiple Scale State-Space Model for Soil Moisture The key ideas behind converting the model (30) to a multiple scale state-space model are as follows. 1) At the finest scale of interest, wherever soil-moisture measurements are available, the model state is completely determined by these measurements. 2) At the finest scale of interest, wherever there is a lack of soil-moisture measurements, estimates based on AWC provide a reasonable approximation. Using these assumptions, the model (30) can be cast into a as multiple scale state-space format with given below (31) as in (9). The domain mean can be where estimated outside of the multiple scale model using appropriate estimation from all available measurements. The measurement equation for the finest scale is given by (32) where
is an indicator function if soil-moisture measurement is available (33) if soil-moisture measurement is not available if soil-moisture measurement is available if soil-moisture measurement is not available (34)
is the error in the measurement of , and is the using the AWC relationship. For error in the estimate of all other scales, the measurement equation is (35) AWC information is not incorporated at coarser scales to avoid the need to scale it up. In the estimation procedure, this information is implicitly propagated to larger scales during the upward pass.
The objective of this section is to demonstrate the applicability of the multiple scale state-space soil-moisture model (31) with measurement equations (32)–(35) for the estimation and conditional simulation of soil-moisture under the following scenarios: 1) estimation and conditional simulation of missing values; 2) estimation and conditional simulation with subsampled values; 3) estimation and conditional simulation of soil moisture outside the measurement domain, where only AWC information is available; 4) estimation and conditional simulation at scales finer than those at which measurements are available. The observation scales are chosen to represent the scales of satellite footprint. Both June 10 and 18, 1992, data sets are considered to illustrate the applicability of the model for wet and dry conditions. For the soil moisture field shown in Fig. 6(a), the model and parameters were estimated as for June 10, 1992. The measurement errors were specified as 3.3% of the measured volumetric soil moisture values for the 64 grid from the remotely sensed data [33]. Data on a 64 original data set were extracted [see Fig. 11(a)] to estimate missing values within the domain of analysis. Fig. 11(b) and (c) show the estimated field and the estimation errors, respectively. Notice that the estimation error is nonuniform since the measurement errors were specified as a function of the soil-moisture values. Also, the estimation errors are large in regions with a lack of data. One realization of the conditionally simulated field is shown in Fig. 11(d). A simulated field looks only marginally more variable than the estimated (smooth) field since the data are quite dense. However, in the large region with no data, the simulation is better able to depict the variability. Estimation and conditional simulation with subsampled values is demonstrated in Fig. 12. Notice the nonhomogeneous distribution of the simulated error field, which has smaller values in the proximity of locations with measurements, and and vice-versa. Consequently, the model parameters are location dependent. As evident, the simulated field depicts variability that is closer to those observed in the original field [compare with 11(a)]. Estimation and conditional simulation of soil moisture outside the measurement domain using AWC information is shown in Fig. 13. Notice that both the estimated and simulated field conform to the expected dependence on the soil properties, as characterized through the AWC. Again, the simulated field shows variability that is closer to the expected natural variability. Fig. 14 shows the same analysis for the June 18, 1992, case to demonstrate the skill of the model for the dry scenario. The model performs quite well for the large range of wet through dry scenario and over a fairly wide range of soil properties. Estimation and conditional simulation at scales finer than those at which measurements are available is shown in Fig. 15. 6.4 km , Pixels were averaged to obtain values at 6.4
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
193
(a)
(b)
(c)
(d)
2
Fig. 11. Estimation and conditional simulation of missing values (June 10, 1992): (a) original field on a 64 64 grid (a subset of area shown in Fig. 6). Missing values are indicated as dark areas; (b) estimated field; (c) estimation error; and (d) conditionally simulated field.
(a)
(b)
(c)
(d)
Fig. 12. Estimation and conditional simulation of soil-moisture field from a sparsely sampled data (June 10, 1992): (a) original field shown in Fig. 11(a) subsampled at 5% threshold. Dark areas indicate missing values; (b) estimated field; (c) simulated error field; and (d) conditionally simulated field.
as shown in Fig. 15(a). This resolution is of the order of magnitude of a satellite footprint. Using the large-scale averages and the AWC information, estimation and conditional simulation were obtained at a 200 200-m scale, as shown in Fig. 15(c) and (d), respectively. Comparing these with Fig. 11, we see that the estimates and simulations are quite realistic and
conform to the heterogeneity dictated by the soil-hydrologic properties. These results demonstrate that a simple mean-differenced fractal model under the multiple scale paradigm can provide realistic simulations of soil moisture and other such fields and at the same time account for the deterministic hetero-
194
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
(a)
(b)
(c)
(d)
Fig. 13. Estimation and conditional simulation of soil moisture outside the measurement domain where only AWC information is available (June 10, 1992): (a) original field at 256 256 grid. Notice that the actual measurements lie on a subset of the domain; (b) available water capacity field for the domain obtained from the STATSGO database; (c) estimated field; and (d) conditionally simulated field.
2
(a)
(b)
(c)
(d)
Fig. 14. Estimation and conditional simulation of soil moisture outside the measurement domain where only AWC information is available (June 18, 1992): (a) original field at 256 256 grid. Notice that the actual measurements lie on a subset of the domain; (b) simulated error field. Notice larger error magnitudes in regions with no measurements; (c) estimated field; and (d) conditionally simulated field.
2
geneity dictated by the underlying passive field. There are some limitations, however. For example, notice the visually “blocky” nature of the generated fields. This is an artifact of the tree structure, in which we can have two nodes that are closely located in space but not on the tree. At least two
approaches are available to address this. First, we may use the ensemble average of several fields with the root node located randomly. Although this produces visually better fields for the estimation algorithm, it cannot be used for the simulation since the ensemble average of the error terms will tend to
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
195
(a)
(b)
(c)
(d)
Fig. 15. Estimation and conditional simulation at scales finer than those at which measurements are available: (June 10, 1992) (a) large-scale average field; (b) simulated error field [see (17)]; (c) estimated field; and (d) conditionally simulated field. Compare these with those in Fig. 11.
zero. Alternatively, overlapping tree models that remove this blocky artifact have been developed [9]. Issues related to the application of this model for conditional simulation are currently being pursued. VI. CONCLUSIONS Estimation of a process and characterization of its variability using measurements at multiple scales is a problem of considerable interest in several areas of geophysics. Elegant estimation techniques, based on multiresolution trees, to combine measurements across scales were developed by Chou et al. [5]–[7]. Due to the very nature of the optimization, the estimation is a smoothing process that may not provide a good representation of the variability, particularly in regions where there are no observations. A complementary conditional simulation technique is developed that allows us to construct synthetic fields that are representative of the intrinsic variability of the process. This uses a multiple scale model, whose parameters can be explicitly computed, for the estimation error process. The conditional model reduces to the unconditional simulation model in the absence of measurements, as should be expected. In order to apply the estimation and conditional simulation to the problem of estimating soil moisture, a mean-differenced fractal model for soil moisture is proposed. The mean accounts for the dependence of soil moisture on the hydrologic properties, and the fluctuations about this mean are modeled as fractional Gaussian noise. This model is then cast in a multiple scale framework and applied to problems of estimation and simulation to the following scenarios: domain with missing values, sparsely sampled data, in domain outside of where
measurements are available, and at scales smaller than at which measurements are available. The simulated fields can find several applications. For example, there is considerable interest in studying the impact of soil moisture variability on atmospheric circulation. Modeling studies using several synthetic fields (conditioned on data) can allow us to assess the sensitivity of the atmospheric response to soil moisture.
APPENDIX MULTIPLE SCALE KALMAN FILTERING To make the paper self-contained, the following provides a summary of the multiple scale Kalman filtering algorithm developed by Chou et al. [1994] (see also [9]). For details, refer to the original publications. Let us assume that the dimensions of state and observation and , respectively. The multiple vectors are scale downward (coarse-to-fine-scale) model and observations are given by [see (1) and (8)] (36) (37)
Define the state and measurement covariance matrices, respectively, as (38)
196
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 1, JANUARY 1999
and .. .
.. .
..
.
.. .
(39)
b) One step prediction: Moving up to the parent node, we apply the Kalman filter prediction to get predictions from each child node using (45)
is diagonal but not a multiple of the identity, Here indicating each measurement component can have a different variance. Corresponding to the above downward model the upward (fine-to-coarse-scale) model is [see (5)] (40) is an uncorrelated sequence with variance [see
where (7)]
(46) (47) [see (15) and (16)]. c) Merge step: Note that each node gets predictions from each of the child nodes. These are then merged to obtain a single prediction using
E (48)
(41) We assume that each node has children. We denote by , the th child node of for . Also define predicted value of using the estimate of ( ) of ; child node predicted value of after merging the predictions of the child nodes of ; updated value of using and ; the measurement estimated value of after smoothing during the downward sweep. , , , and Define the error variances analogously. The estimation proceeds in the following steps. • Initialization: Assign the following prior values at corresponding to the finest scale node
(49) The upward sweep terminates when the recursion of update, predict, and merge reaches the root node and we . obtain the estimate • Downward Sweep: The information is propagated downward after the upward sweep is complete. The estimators are (50) (51) where (52)
where is the prior error variance, i.e., solution of (4), at the node . • Upward sweep: a) Measurement update: from The predicted (and merged) values the child nodes are combined with the measurements to get estimates of both the state vectors and the error variances using
(42) (43) where the Kalman gain matrix
is given by
(44)
see [(15) and (16)]. Both (46) and (51) indicate that the estimate at a particular node in the downward sweep is equal to the sum of its estimate in the upward sweep and the difference in the estimates of the parent node in the downward and upward sweep (value in the brackets) weighted by a coefficient. ACKNOWLEDGMENT The author would like to thank T. Jackson for making the soil moisture data available and P. Feiguth for making the multiscale estimation programs available in the public domain. REFERENCES [1] M. Basseville, A. Benveniste, and A. S. Willsky, “Multiscale autoregressive processes—Part 1,” IEEE Trans. Signal Processing, vol. 40, pp. 1915–1934, Nov. 1992. [2] , “Multiscale autoregressive processes—Parts 1 and 2,” IEEE Trans. Signal Processing, vol. 40, pp. 1935–1954, Nov. 1992.
KUMAR: CHARACTERIZING VARIABILITY OF SOIL MOISTURE AT VARIOUS SCALES
[3] A. Benveniste, R. Nikoukhah, and A. S. Willsky, “Multiscale system theory,” IEEE Trans. Circuits Syst., vol. 41, pp. 2–15, Jan. 1994. [4] K. J. Beven and M. J. Kirkby, “A physically based variable contributing area model of basin hydrology,” Hydrol. Sci. Bull., vol. 24, no. 1, pp. 43–69, 1979. [5] K. C. Chou, “A stochastic modeling approach to multiscale signal processing,” Ph.D. dissertation, Mass. Inst. Technol., Cambridge, May, 1991. [6] K. C. Chou, A. S. Willsky, and A Benveniste, “Multiscale recursive estimation, data fusion, and regularization,” IEEE Trans. Automat. Contr., vol. 39, pp. 464–478, Mar. 1994. [7] K. C. Chou, A. S. Willsky, and R. Nikoukhah, “Mutiscale systems, Kalman filters, and Riccati equations,” IEEE Trans. Automat. Contr., vol. 39, pp. 479–492, Mar. 1994. [8] C. V. Deutsch and A. G. Journel, GSLIB: Geostatistical Software Library and User’s Guide. Oxford, U.K.: Oxford Univ. Press, 1992, p. 340. [9] P. W. Fieguth, “Application of multiscale estimation to large scale multidimensional imaging and remote sensing problems,” Ph.D. dissertation, Mass. Inst. Technol., Cambridge, May, 1995. [10] C. R. Hicks, Fundamental Concepts in the Design of Experiments. New York: CBS, 1982, p. 425. [11] T. J. Jackson, D. M. Le Vine, C. T. Swift, T. J. Schmugge, and F. R. Schiebe, “Large area mapping of soil moisture using the ESTAR passive microwave radiometer in Washita’92,” Remote Sens. Environ., vol. 53, pp. 27–37, 1995. [12] S. E. Hollinger and S. A. Isard, “A soil moisture climatology of Illinois,” J. Climate, vol. 7, no. 5, pp. 822–833, 1994. [13] T. J. Jackson, “Surface soil moisture sampling,” Hydrology Data Report, vol. 93, no. I, T. J. Jackson and F. R. Schiebe, Eds. Nat. Agricultural Water Quality Lab., Durant, OK, 1993. [14] A. G. Journel, “Geostatistics for conditional simulation of ore bodies,” Econ. Geol., vol. 69, pp. 673–687, 1974. , Fundamentals of Geostatistics in Five Lessons. Washington, [15] DC: Amer. Geophys. Union, 1989, p. 40. [16] A. G. Journel and Ch. J. Huijbregts, Mining Geostatistics. Boston, MA: Academic, 1978, p. 600. [17] M. R. Luettgen, “Image processing with multiscale stochastic models,” Ph.D. dissertation, Mass. Inst. Technol., Cambridge, May 1993. [18] M. R. Luettgen and A. S. Willsky, “Likelihood calculation for a class of multiscale stochastic models, with application to texture discrimination,” IEEE Trans. Image Processing, vol. 4, pp. 194–207, Feb. 1995. , “Multiscale smoothing error models,” IEEE Trans. Automat. [19] Contr., vol. 40, pp. 173–175, Jan. 1995. [20] D. R. Maidment, Handbook of Hydrology. New York: McGraw-Hill, 1993. [21] B. Mandelbrot and V. Ness, “Fractional Brownian motions, fractional
197
noises and applications,” SIAM Rev., vol. 10, no. 4, pp. 422–437, 1968. [22] H. E. Rauch, F. Tung, and C. T. Striebel, “Maximum likelihood estimates of linear dynamic systems,” AIAA J., vol. 3, no. 8, pp. 1445–1450, 1965. [23] I. Rodriguez-Iturbe, G. K. Vogel, R. Rigon, D. Entekhabi, F. Castelli, and A. Rinaldo, “On the spatial organization of soil moisture fields,” Geophys. Res. Lett., vol. 22, no. 20, pp. 2757–2760, 1995. [24] P. J. Rousseeuw, “Least median of squares regression,” J. Amer. Stat. Assoc., vol. 79, pp. 871–878, 1984. [25] Soil Conservation Service (SCS), State Soil Geographic Data Base (STATSGO) Data Users Guide., U.S. Dept. Agriculture, Soil Conservation Service, 1993, Misc. Publ. 1492, p. 88. [26] E. R. Stofan, D. L. Evans, C. Schmullius, B. Holt, J. J. Plaut, J. van Zyl, S. D. Wall, and J. Way, “Overview of results of spaceborne imaging radar-C, X band synthetic aperture radar (SIR-C/X-SAR),” IEEE Trans. Geosci. Remote Sensing, vol. 33, pp. 817–828, July 1995. [27] A. F. B. Tompson, R. Abanou, and L. Gelhar, “Application and use of the three dimensional tuning bands random field generator: Single realization problems,” Dept. Civil Eng., Mass. Inst. Technol., Cambridge, Rep. 313, 1987. [28] G. Verghese and T. Kailath, “A further note on backward Markovian models,” IEEE Trans. Inform. Theory, vol. 25, pp. 121–124, Jan. 1979. [29] R. F. Voss, “Fractals in nature: From characterization to simulation,” Science of Fractal Images, H.-O. Peitgen and D. Saupe, Eds. New York: Springer-Verlag, 1988. [30] G. W. Wornell and A Karhunen-Loeve, “Like expansion for 1=f processes,” IEEE Trans. Inform. Theory, vol. 36, pp. 859–861, July 1990. [31] G. W. Wornell and A. V. Oppenheim, “Wavelet-based representations for a class of self-similar signals with application to fractal modulation,” IEEE Trans. Inform. Theory, vol. 38, pp. 785–800, June 1992. [32] M. R. Luettgen, personal communication, 1996. [33] T. J. Jackson, personal communication.
Praveen Kumar received the M.S. degree from Iowa State University, Ames, and the Ph.D. degree from the University of Minnesota, Minneapolis–St. Paul, in 1993, respectively. He was with the Universities Space Research Association, NASA Goddard Space Flight Center, Greenbelt, MD, from 1993 to 1995. He is currently with the University of Illinois, Urbana. His research interests include large-scale hydrologic processes, with particular emphasis on problems related to multiple scale interactions among subprocesses. He is currently the Associate Editor of Water Resources Research.
