Linear Combinations of Space-Time Covariance Functions and ...

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 3, MARCH 2005

857

Linear Combinations of Space-Time Covariance Functions and Variograms Chunsheng Ma

Abstract—The difference or a linear combination of two spacetime covariance functions (or variograms) is not necessarily a valid covariance function (or variogram). In general, there seems no simple condition that makes a linear combination permissible, unless its coefficients are non-negative. The permissibility is investigated in this paper for the linear combination of two spatial or spatio-temporal covariance functions (or variograms) isotropic in space so that we obtain flexible classes of spatial or spatio-temporal covariance functions with various properties such as long-range dependence and having different signs. Index Terms—Covariance, Fourier transform, intrinsically stationary, isotropic, long-range dependence, power-law decay, spectral density, stationary, variogram.

I. INTRODUCTION

S

TOCHASTIC models that describe how processes vary across time, space, or space and time are useful in geophysical, informational, and environmental sciences. Statistical and probabilistic techniques of signal processing, time series analysis, or spatial statistics provide important tools for modeling temporal or spatial data as well as numerical simulation. See, for instance, [2]–[4], [6], [8], [10], [13], [17], [22], [27], [30], [33], and [36]. The demand for modeling space-time data has led to many recent advances in the development of spatio-temporal variograms and covariance models, for which we refer to [5], [7], [11], [12], [14], [15], [20], [21], [23], [25], [26], [34], and [37], among others. is a real-valued random Suppose that field over the space index and the time index , where and . Two commonly used measures of space-time interaction and dependence are the covariance function and variogram. The covariance function is defined by

and the variogram or structure function is half the variance of the difference var

Manuscript received August 18, 2003; revised April 5, 2004. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Joseph Tabrikian. The author is with the Department of Mathematics and Statistics, Wichita State University, Wichita, KS 67260-0033 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TSP.2004.842186

is a cone since is a covariance , are non-negative constants and function whenever are covariance functions, and so is . However, the difference of the set of all variograms on two covariances (or variograms) is not necessarily a covariance function (or variogram). The aim of this paper is to investigate the conditions under which a linear combination of covariance functions (or variograms) is a valid covariance function (or variogram). There are several reasons that motivate us to investigate the validity of the difference or a linear combination of two covariances (or variograms). First, it is often desirable [32] to have a covariance function that may be negative or oscillate from positive to negative values as it tends to zero when the Euclidean approaches infinity, but spatial covariance models frenorm quently used in practice [27] are non-negative. Second, it would be better for statistical inference and kriging if we are able to determinate the range of the parameter such that

The set of all covariance functions on

(1.1) is a permissible covariance. Third, we want to know how the parameter affects the domain of (1.1), which may be valid only in lower spatial dimensions if one of its coefficients is negative. Fourth, there is a great demand to construct long-range dependent space-time models since a rapidly expanding empirical literature has found evidence of long-range dependence in many spatial and time series data [12], [24], [29], [35]. The most important parameter in our framework is , which determines when the linear combination is a valid model and in which dimension it is permissible. However, there seem no simple conditions that make a linear combination like (1.1) ac. An ceptable, except for a convex combination where attempt is made in this paper for the linear combination of two spatial or spatio-temporal covariances (or variograms) isotropic in space. A typical way to obtain an anisotropic covariance or by , where variogram from an isotropic one is to replace is a non-negative definite matrix. A more general approach is the spatial deformation method [28], [30], which replaces the by , where Euclidean distance is a deformation of the geographic coordinate system. Alternatively, evolutionary processes can be obtained through convolutions of local stationary processes [9], [16]. Section II considers linear combinations of purely spatial, isotropic covariance functions in all dimensions. A random field or its covariance function is said to

1053-587X/$20.00 © 2005 IEEE

858

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 3, MARCH 2005

be isotropic if is only a function of the Euclidean dis, in which case, we simply write as tance . Isotropic covariance functions are the most popularly used models in practice [4], [6], [10], [33], [36], with many important demonstrations and useful properties. A continuous isotropic covariance function for all dimensions is of the form [31]

where is a completely monotone function on , which by definition is a continuous function on with derivatives of all orders on , and for all or, equivalently, is the Laplace transform of a bounded and non, , decreasing function (1.2) Isotropy can be viewed as an invariance property under the transformations of rotations and reflections. Two more general invariance properties are (weak or second-order) stationarity and intrinsic stationarity. A space-time random field or its covariance is said to be stationary in space and time, if and is only a function of and , for in which case, we simply write . It is stationary in space (or stationary in time), (or ) and if is only a function of (or ). In terms of the variogram, intrinsic stationarity is weaker than or its stationarity. A random field is said to be intrinsically stationary variogram is a constant and in space and time if is only a function of and , in which case, is simply written as . In Section III, we consider linear combinations of purely spatial, isotropic variograms in all dimensions, for which the concept of the Bernstein function is needed. A non-negative, conon with is called a tinuous function Bernstein function [1] if it is the integral of a completely mono, namely tone function

(1.3) where , is nondecreasing, and exists. In terms of the Bernstein function, a continuous, isotropic varifor all dimensions is known [31] to be of a simple ogram form

where is a Bernstein function on with . By linear combinations of purely spatial, isotropic variograms

in all dimensions, we obtain variograms limited to certain dimensions in Section III. Section IV studies linear combinations of space-time covariance functions and derives spatio-temporal models with powerlaw decay spatial or temporal margin. Theorems are proved in Section V. and are distinct Throughout the paper, we assume that positive constants with , is a completely monotone , and is a Bernstein function on function on with . II. LINEAR COMBINATIONS OF ISOTROPIC COVARIANCE FUNCTIONS In this section, we investigate linear combinations of continuous, isotropic covariance functions in all dimensions. It is found out that the resultant covariance may be just available in certain finite dimensions if one of the coefficients of the linear combination is negative. , is often referred in the The function literature as a Gaussian covariance model [4]. It is infinitely differentiable and available for all dimensions . Theorem 1 provides the permissible condition for a linear combination of two Gaussian covariances as well as for a linear combination of two isotropic covariance functions in all dimensions. Theorem 1: i) The function

is a stationary covariance function on is a constant with

(2.1) if and only if

(2.2) ii)

If

satisfies (2.2), then (2.3)

iii)

is a stationary covariance function on . If, in addition, is of the form (1.2) with , then a necessary condition for (2.3) to be a stationary covariance function on is

The lower bound of (2.2) depends highly on the dimensional parameter , in which case, (2.1) is not always positive, and as approaches infinity, the lower bound of (2.2) tends to zero. The interval [0,1] is obviously included in the interval whose end points are the bounds of (2.2). The spectral density function of when and not monotone (2.1) is decreasing in . Similar conclusions can be when drawn for the model (2.3), whose spectral density exists under . the assumption of a finite integral

MA: LINEAR COMBINATIONS OF SPACE-TIME COVARIANCE FUNCTIONS AND VARIOGRAMS

Example

1: The logistic function , is a scale mixture of Gaussian distri, is butions. Thus, a covariance function for all dimensions , and when (2.2) holds, is a covariance function on . In the particular case where equals the lower bound of (2.2), decreases from positive to negative, goes reaches its minimum, and then increases to zero, as from zero to infinity. the modified Bessel funcExample 2: Denote by tion of the second kind of order . The function , , is completely monotone [18]. It follows from Theorem 1 ii) that

Only if this case, (2.4) reduces to

859

can (2.4) be differentiable. In

(2.6) which is a mixture of the spatial autoregression of order 2 proposed by [32] as a three-dimensional (3-D) turbulence model and by [3] for meteorological applications. The spatial autoregression of a general order is recently introduced in [25]. In parand in (2.6) yields ticular, letting

which is a mixture of the spatial autoregression of order 2 with the same roots. The function (2.4) is essentially a special case of (2.3) once we rewrite the former as is a covariance function on

if

The model (2.3) is actually a mixture of (2.1). Likewise, (2.4) below is a mixture of the spatial autoregressive and moving averaging covariances

whose univariate projection is studied in [24], where Theorem 2: i) The function

where is a completely monotone function on since is a completely monotone function and is a . From this observation, we can see Bernstein function on that (2.2) is a sufficient but not always a necessary condition for (2.3) to be a covariance function on . Next, we illustrate how to construct spatial long-range dependent covariances from (2.4) via Bernstein functions. Example 3: For a positive constant , the function

. is completely monotone. Substituting it in (2.4) produces a power-law decay covariance function (2.4)

is a stationary covariance function on stant with

if

is a con-

(2.5)

Under an additional assumption that is of the form and (1.2) with , (2.5) is also a necessary condition for (2.4) to be a covariance function on . Just like (2.2), the lower bound of (2.5) depends highly on the dimensional parameter so that the resultant covariance is not always positive, and it tends to zero as approaches infinity. In contrast, the upper bound of (2.5) does not depend on , which results in a covariance function in all dimensions and is nonnegative for all spatial lag .

For a constant between 0 and 1, , , and , are Bernstein functions, and so are , , , and , , and so on. Many completely monotone functions are thus be formed; for instance

ii)

and

860

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 3, MARCH 2005

is a variogram on

if

According to a remarkable result of [31], if is a comand is an intrinsically pletely monotone function on is a stationary covaristationary variogram on , then ance function on . One can apply this result and Theorem 3 to derive many new stationary spatial covariance functions. Moreover, combining the above Theorem 3 with [21, Th. 3] results in spatio-temporal stationary covariance models. IV. LINEAR COMBINATIONS OF SPATIO-TEMPORAL COVARIANCE FUNCTIONS Fig. 1.



Plot of covariance function in Example 3 iv) on

= 1,  = 3, = 1=2, and d = 2.

, where  = 0:4,

Accordingly, the following slowly decaying covariance functions are derived from Theorem 2. i) , , where . ii) , , where satisfies (2.5). iii) , , where . iv) , , where satisfies (2.5). A 3-D plot of the covariance function defined in iv) versus is illustrated in Fig. 1.

We now derive some spatio-temporal covariance functions via linear combinations, which are formulated by using the cosine transform method [23]. Theorem 4 gives spatio-temporal stationary covariance functions with the power-law decay temporal margin, and Theorem 5 gives those with the power-law decay spatial margin. Theorem 4: i) The function

(4.1) is a spatio-temporal stationary covariance on and only if satisfies

if

(4.2) ii)

The function

III. LINEAR COMBINATIONS OF ISOTROPIC VARIOGRAMS Linear combinations of purely spatial, isotropic variograms in all dimensions are studied in this section, for which the key concept is the Bernstein function. Theorem 3: i) When inequality (2.2) holds, the function

(4.3) is a stationary covariance on

if and only if (4.4)

(3.1)

ii)

is an intrinsically stationary variogram on Under inequality (2.5), the function

.

(3.2) is an intrinsically stationary variogram on . As an example, consider the Bernstein function , , where . By Theorem 3 ii)

Both (4.1) and (4.3) are spatio-temporal covariances on for every positive integer since neither (4.2) nor (4.4) involve the dimensional parameter . They possess the same spatial margin, which is defined as the covariance of the purely spais fixed tial random field

The temporal margin of (4.1) is, assuming

,

MA: LINEAR COMBINATIONS OF SPACE-TIME COVARIANCE FUNCTIONS AND VARIOGRAMS

which is power-law decay, and so is the temporal margin of (4.3)

The spatial margin of (4.1) [or (4.3)] is always non-negative on , but its temporal margin can assume negative values when is equal to the left bound of (4.2) [or (4.4)]. As a result, (4.1) or (4.3) takes positive and negative values when or . Theorem 5: i) The function

861

Example 5: For a constant between 0 and 1, the function , is completely monotone. Therefore, under the assumption of (4.2)

is a spatio-temporal covariance function on with the power-law decay temporal margin, and under inequality (2.5)

(4.5)

ii)

is a spatio-temporal stationary covariance on satisfies inequality (2.5). The function

if is a spatio-temporal covariance function on power-law decay spatial margin.

with the

V. PROOFS (4.6) is a spatio-temporal stationary covariance on satisfies (2.2). The temporal margins of (4.5) and (4.6) are the same

Proof of Theorem 1: i) The Fourier transform of

if

The spatial margins of (4.5) and (4.6) are power-law decay and can assume negative values when equals the lower bound of (2.5) or (2.2), in which case, (4.5) or (4.6) takes positive and negative values. In contrast to (4.1), (4.5) is a spatio-temporal covariance on but not for all dimensions , unless is between 0 and . Similar contrasts apply to (4.3) and (4.6). Example 4: For a positive constant , taking , , in (4.1) and (4.5), respectively, yields

If is a covariance function, then its spectral must be non-negative for all . In density particular, (2.2) results from

and

On the other hand, suppose that satisfies for every (2.2). We will show that . This is true when is between 0 and 1. , we obtain In case , and thus, for

and

which are covariance functions on decay in space and time.

defined by (2.1) is

with the power-law

ii)

Being a mixture of the covariance (2.1), the function (2.3) is a covariance on .

862

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 3, MARCH 2005

iii)

When , the Fourier transform of (2.3) exists and is given by

In case

, it follows that

For (2.3) to be a covariance, it is necessary that

ii)

or Proof of Theorem 2: i) It suffices to show that for each fixed variance on rewritten as

where on

if

,

is a cosatisfies (2.5), since (2.4) can be

is a nondecreasing and bounded function . Notice that for ,2

where transform of

If (2.4) is a covariance, its spectral density has to be . In particular, from , non-negative for all , or equivalently, we obtain , and obtain from

. The Fourier is

To show that (2.5) is a sufficient condition for to be a covariance on , or equivalently, for be non-negative in , we consider two cases. , or In case , we obtain

Note that for

Since is the Laplace transform of a bounded on , and nondecreasing function , and , the Fourier transform of (2.4) is

to

Proof of Theorem 3: We prove Part i) by a check of negative defined by (3.1) only. A proof definiteness of the function for Part ii) can be similarly performed. by (1.3), we obtain an integral representaExpressing tion for (3.1):

MA: LINEAR COMBINATIONS OF SPACE-TIME COVARIANCE FUNCTIONS AND VARIOGRAMS

For any integer , spatial lags , and real numbers subject to , we have

ii)

863

The only if part follows by looking at the temporal margin of (4.3), assuming

It is of the form (2.3) with , , and thus, by Theorem 1 iii), it is necessary that . To show , we calculate the spectral density function of

which must be non-negative for all all

where the last inequality follows from the positive definiteness , of the function for every which by Theorem 1 i) is a spatial covariance on , provided that satisfies inequality (2.2). Proof of Theorem 4: i) The only if part follows simply by looking at the temporal margin of (4.1), assuming

or

We obtain from the last inequality by letting , noting that

which is of the form (2.4) on the real line with

By Theorem 2 ii), (4.2) is a necessary condition for to be a temporal covariance function. The approach of [23] is used to prove the if part. , the function First, we show that for every fixed

is a spatio-temporal covariance on from [23, Corol. 2.1] once we express

. This follows as

where for every fixed , is a temporal covariance function on the real line under the condition of (4.2), according to [24, Lemma 1]. is a spatio-temporal covariance as a Now, mixture of

. Hence, for

is the Laplace transform of infinitely divisible probability distributions [19] so that it tends to zero as approaches infinity. The if part is proved in a way similar to that of Part i). To this end, it suffices to express (4.3) as a mixture of

where

is a spatio-temporal covariance on , since for every fixed

for every fixed , by Theorem 1 i), is a temporal covariance function on the real line if (4.4) holds. Proof of Theorem 5: We give a proof of Part i) only, while Part ii) can be analogously derived. defined by (4.5) is a mixture of Notice that

864

where for

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 53, NO. 3, MARCH 2005

,

is given by

By [23, Corol. 2.4], is a spatio-temporal covariance on , since it can be rewritten as

where for every fixed , is the covariance for a spatial autoregression of order 2 under the condition (2.5). REFERENCES [1] C. Berg and G. Forst, Potential Theory on Locally Compact Abelian Groups. New York: Springer, 1975. [2] G. E. P. Box, G. M. Jenkins, and G. C. Reinsel, Time Series Analysis: Forecasting and Control, Third ed. Englewood Cliffs, NJ: PrenticeHall, 1994. [3] C. E. Buell, “Correlation functions for wind and geopotential on isobaric surfaces,” J. Appl. Meteorol., vol. 11, pp. 51–59, 1972. [4] J.-P. Chilès and P. Delfiner, Geostatistics: Modeling Spatial Uncertainty. New York: Wiley, 1999. [5] G. Christakos, Modern Spatiotemporal Geostatistics. Oxford, U.K.: Oxford Univ. Press, 2000. [6] N. Cressie, Statistics for Spatial Data. New York: Wiley, 1993. [7] N. Cressie and H. C. Huang, “Classes of nonseparable, spatio-temporal stationary covariance functions,” J. Amer. Statist. Assoc., vol. 94, pp. 1330–1340, 1999 [Correction: vol. 96, p. 784, 2001]. [8] P. M. Fishman and D. L. Snyder, “The statistical analysis of spacetime point processes,” IEEE Trans. Inf. Theory, vol. IT-22, pp. 257–274, 1976. [9] M. Fuentes, “Spectral methods for nonstationary spatial processes,” Biometrika, vol. 89, pp. 197–210, 2002. [10] G. Gaspri and S. E. Cohn, “Construction of correlation functions in two and three dimensions,” Q. J. Roy. Meteorol. Soc., vol. 125, pp. 723–757, 1999. [11] A. E. Gelfand, H. J. Kim, C. F. Sirmans, and S. Banerjee, “Spatial modeling with spatially varying coefficient processes,” J. Amer. Statist. Assoc., vol. 98, pp. 387–396, 2003. [12] T. Gneiting, “Nonseparable, stationary covariance functions for spacetime data,” J. Amer. Statist. Assoc., vol. 97, pp. 590–600, 2002. [13] P. Goovaerts, Geostatistics for Natural Resources Evaluation. Oxford, U.K.: Oxford Univ. Press, 1997. [14] T. C. Haas, “Statistical assessment of spatio-temporal pollutant trends and meteorological transport models,” Atmos. Env., vol. 32, pp. 1865–1879, 1998. [15] , “New systems for modeling, estimating, and predicting a multivariate spatio-temporal process,” Env., vol. 13, pp. 311–332, 2002. [16] A. Hanssen and L. L. Scharf, “A theory of polyspectra for nonstationary stochastic processes,” IEEE Trans. Signal Process., vol. 51, no. 5, pp. 1243–1252, May 2003.

[17] J. O. Hinze, Turbulence, Second ed. New York: McGraw-Hill, 1975. [18] M. E. H. Ismail, “Bessel functions and the infinite divisibility of student t-distribution,” Ann. Prob., vol. 5, pp. 582–585, 1977. [19] M. E. H. Ismail and D. H. Kelker, “Special functions, Stieltjes transforms and infinite divisibility,” SIAM J. Math. Anal., vol. 10, pp. 884–901, 1979. [20] C. Ma, “Spatio-temporal covariance functions generated by mixtures,” Math. Geol., vol. 34, pp. 965–975, 2002. , “Spatio-temporal stationary covariance models,” J. Multivariate [21] Anal., vol. 86, pp. 97–107, 2003. [22] , “Power-law correlations and other models with long-range dependence on a lattice,” J. Appl. Prob., vol. 40, pp. 690–703, 2003. , “Families of spatio-temporal stationary covariance models,” J. [23] Statist. Plann. Infer., vol. 116, pp. 489–501, 2003. [24] , “Long memory continuous-time correlation models,” J. Appl. Prob., vol. 40, pp. 1133–1146, 2003. [25] , “Spatial autoregression and related spatio-temporal models,” J. Multivariate Anal., vol. 88, pp. 152–162, 2004. [26] P. Northrop, “A clustered spatial-temporal model of rainfall,” Proc. Roy. Soc. London A, vol. 454, pp. 1875–1888, 1998. [27] D. S. Oliver, “Moving average for Gaussian simulation in two and three dimensions,” Math. Geol., vol. 27, pp. 939–960, 1995. is second-order sta[28] O. Perrin and W. Meiring, “Non-stationarity in tionarity in ,” J. Appl. Prob., vol. 40, pp. 815–820, 2003. [29] A. P. Petropulu, J.-C. Pesquet, X. Yang, and J. Yin, “Power-law shot noise and its relationship to long-memory -stable processes,” IEEE Trans. Signal Process., vol. 48, no. 7, pp. 1883–1891, Jul. 2000. [30] P. Sampson and P. Guttorp, “Nonparametric estimation of nonstationary spatial covariance structure,” J. Amer. Statist. Assoc., vol. 87, pp. 108–119, 1992. [31] I. J. Schoenberg, “Metric spaces and completely monotone functions,” Ann. Math., vol. 39, pp. 811–841, 1938. [32] I. P. Shkarofsky, “Generalized turbulence space-correlation and wavenumber spectrum-function pairs,” Can. J. Phys., vol. 46, pp. 2133–2153, 1968. [33] M. L. Stein, Interpolation of Spatial Data: Some Theory for Kriging. New York: Springer-Verlag, 1999. [34] H. S. Wheater, V. S. Isham, D. R. Cox, R. E. Chandler, A. Kakou, P. J. Northrop, L. Oh, C. Onof, and I. Rodriguez-Iturbe, “Spatial-temporal rainfall fields: modeling and statistical aspects,” Hydrol. Earth Syst. Sci., vol. 4, pp. 581–601, 2000. [35] P. Whittle, “Topographic correlation, power-law covariance functions, and diffusion,” Biometrika, vol. 49, pp. 305–314, 1962. [36] A. M. Yaglom, Correlation Theory of Stationary and Related Random Functions. New York: Springer, 1987. [37] S. Zhou and G. B. Giannakis, “Optimal transmitter eigen-beamforming and space-time block coding based on channel correlations,” IEEE Trans. Inf.. Theory, vol. 49, no. 7, pp. 1673–1690, Jul. 2003.

Chunsheng Ma received the Ph.D degree from the University of Sydney, Sydney, Australia, in 1997. From 1997 to 1999, he was a teaching postdoctoral fellow in the University of British Columbia, Vancover, BC, Canada, and in August 1999, he joined Wichita State University, Wichita, KS, as a faculty member. His research areas have been broad in statistics and probability and the current research interests are in time series analysis, random fields, and spatio-temporal statistics.