An Efficient Technique for High Correlation Estimation - IEEE Xplore

654

IEEE TRANSACTIONS ACOUSTICS, ON

SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-35, NO. 5 , MAY 1987

An Efficient Technique for High Correlation Estimation GIOVANNI JACOVITTI

AND

ROBERTO CUSANI,

MEMBER, IEEE

Abstract-Two sum-based estimators are proposed as fast, accurate From {x ( i ) } a set of difference processes for each j methods for estimating the normalized acf of stationary processes.For can be defined as Gaussian processes, their performances are evaluated in terms of bias and covariance, and are compared to the ones of the conventional Di{ z j ( i ) } { x ( i ) - x(i + j ) } . (2) rect estimator and of the Hybrid-Sign and the Polarity Coincidence fast methods. For high correlation values, the proposed estimators exThe variances of these new wide sense stationary, zerohibit the same accuracy as the Direct estimator.

mean processes are

I. INTRODUCTION HE estimation of the autocorrelation function of a process is a common task in most current digital signal processing techniques. For this reason, fast estimation methods, based on lowcost operations, have received special attention in the literature (see, for instance, [2], [8], and the bibliography of [9]).Their effectiveness has been verified in many practical situations. However, some authors have recently pointed out that their loss in accuracy with respect to the conventional Direct method based on correlation products diverges for high correlation estimates [ 3 ] , [ 5 ] ,[6]. This implies that in thesesituations, which occur in spectral moments estimation, autoregressive analysis, time delay measurements, etc., the computational gain is lost because of the need of more samples in order to preserve the desired accuracy. More serious problems arise when a fixed amount of samples is actually available or short-term stationary models of the observed process are involved. In these situations, the accuracy loss of the existing fast techniques may be unacceptable. The aim of this paper is to examine the features of a new fast estimation technique for the normalized acf, designed to operate more efficiently in high correlation environment. Basically, theproposed method consists of estimating the one’s complement of the normalized acf by applying a known fast estimator to a suitably transformed version of the original process. Let us considera wide sensestationary,zero-mean, discrete time process {x ( i ) } characterized by its acf :

T

~ , ( j ) L ~ { x ( i )x ( i + j ) } = a : p ( j )

(1)

where 0; represents the variance and p ( j ) is the normalized acf to be estimated. Manuscript received January 25, 1985; revised October 23, 1986. G . Jacovitti is with the INFOCOM Department, Universitl di Roma “La Sapienza,” Rome, Italy. R. Cusani was with the INFOCOM Department, Universitl di Roma “La Sapienza,” Rome, Italy. He is now with the Dipartimento di Elettronicia, Universitl di Roma “Tor Vergata,” Rome, Italy. IEEE Log Number 8613503.

a;.

E ( z , Z ( i ) ) = 2[R,(O)

-

R,(j)]

= 2d[1 - ~ ( j ) ]

(3)

so that the normalized acf can be expressed as 2

1 OZi

p ( j ) = 1 --7.

2

(4)

0,

Let us now refer in particular to Gaussian processes. In this case, it is well known that [2]

azj =

4;

a,

$E(

r

=

E(

so that (4) can be rewritten as

This is the equation on which the proposed technique is based. 11. THE ABSOLUTE DIFFERENCE AVERAGEESTIMATORS

Let us suppose that the acf of the process under observation can be estimated on thebasis of a single realization (see [ 121), and that N pairs of samples are extracted from the actual realization:

x(hi),x(hi + j )

i = 0, 1 ,

-

*

,N - I

where the positive integer value h represents the distance between the precursors of two consecutive pairs (hopping step). This general situation ranges from the case of contiguous pairs ( h = 1) where all the available data are used to the case h = 03 which is, in practice, equivalent to extracting pairs from independent realizations. Notice that in our case of high correlation estimation, the use of in-

0096-351 8/87/0500-0654$01 .OO @ 1987 IEEE

655

TECHNIQUE FOR HIGHESTIMATION CORRELATION

JACOVITTI CUSANI: AND

termediate values of h avoids the redundant contribution of highly correlated pairs in the estimate. The most usually employed technique for the normalized acf is the Direct estimator, based on sums of products:

and

N- 1

a&>

c x ( h i ) x(hi + j )

a

= rN-l

i=O

N- 1

i=O

71/z*

(8)

i=O

To save computation, the Polarity Coincidence correlator (PC) and the Hybrid-Sign (HS) or “relay” estimators [3] are used in many applications [6], [9]. They are defined as follows: r

. N-I

In [3], the accuracy of the PC estimate is given for the case h = 00. In [ 5 ] , the bias and the covariance of the HS estimates are calculated for every value of the hopping step. These results will be used here for comparison. A j ) is presented in [lo]. The promodified version of posed estimators are based on a directestimate of the expected values in (7):

aHS(

The first one will be referred to as the “full” absolute difference average (ADA) estimator, and the second one as the “reduced” ADA estimator. The full ADA estimate requires 3 N sums (for the first lag, N - 1 extra sums must be accounted for), while the reduced version saves N sums.

111. THE ACCURACY OF THE ADA ESTIMATORS The evaluation of the performances of the proposed estimators moves from a preliminary calculus of the expected values and the covariances of the partial estimates Pj and Qj. It is immediately found from definitions (1 1), (12) and (51, (6)that

E{Q~= } a,

4:

-.

As far as the covariances are concerned, their evaluation is based on the knowledgeof the twofold moment E { 1 u I * I v I } of the zero-mean variables u, z 1. For jointly Gaussian variables, this moment -is deduced from the more general result reported in the Appendix of [4]. We have

. N-1

(17) being The quantity Pj defined above is the so-called average magnitudedifference function (AMDF ), widelyemployed for measuring the pitch period of voiced speech sounds [l]. In particular, we will consider two basic forms:

R,, = a E(u

*

v);

G;

E(u2);

02,

E(2). ( 18)

From the definitions and (17), aquite analysis yields

straightforward

656

IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING. VOL. ASSP-35, NO. 5, MAY 1987

.

N-IN-1

The above calculated moments are bounded to the order 0 ( 1 / N ) if the process memory is limited (the acf has a finite extension a ) . In fact,thedouble summations in (19)-(21) factorize for 1 n - i I exceeding a fixed quantity aj depending on a and j so that they are cancelled out by the products of mean values. The domain I n - i I < ai is only linearly increasing with N so that the overall contributions of the sums must decay as 1/ N , provided that their terms are finite, and they are multiplied by 1 / N 2 . As a first result of the above analysis, the theoretical accuracy of the AMDF estimate (1 1) is calculated for Gaussian processes. The variance is directly obtained by posing j , = j 2 = j in (19). The finite time extension of the acf is a sufficient (and very often verified) condition for the estimate to be consistent. For the limitsituation of uncorrelated pairs of samples, the variance assumes a rather simple form which does not depend on the shape of the acf: lim var ( P j } =

[1

- p(j)].

(22)

h-+m

Using now the calculated covariances, we evaluate the accuracy of the ADA estimators in an approximate way by expanding in series Pj and Qj around their mean values and retaining the terms up to the second order. A detailed discussion of this derivation is reported in the Appendix. For the full ADA estimator, the following bias and co-

variance are obtained: T

b{ $ADA(j))

{ P j } + 6[1

= - 2 [var 4 ox *

var

{Qj}

-

4J2[1

cov (Pi, Q j } ] cov

- d j > I

-~ ( j ) ]

+ O(1/N2)

(23)

{ijADA(jl),fiADA(j2)}

2a

=

7[ I - p ( j l ) ] [ l - ~ ( h ) cov ]

{Qjl? QjZ}

0,

+ 5 J1

- p(j1)

Ji-T'GJ cov {pjl,

pj2)

@X

-

J2[ 1

- P ( ~ I ) ]cov

cov { P j i , Qj2>

{ Qjl,

PjZ} - J2[1 -

+ O(l/N2).

~ ( j 2 ) ]

(24)

The counterparts for the reduced ADA estimator are simply obtained by replacing Qj, Q j l , and Qj, with QO in the above expressions. From the reported results, it is immediately verified that the ADA estimates are asymptotically unbiased and consistent under the hypothesis of time-limited acf. Often, the estimates of the acf are carried out by sequences of N consecutive samples with h = 1 . In this situation, N - j pairs are actually employed for the esti-

JACOVITTI AND CUSANI: TECHNIQUE FOR HIGH CORRELATION ESTIMATION

mate at lag j . The accuracyof the correspondingestimates is then evaluated with the same procedurewith little modification. Finally, it is interesting to calculate thevariances of the ADA estimate for the limit case of independent pairs of samples. The following simple closed forms areobtained: lim var

h-rm

{ hADA(j )}

P 0

4 [1 N

=-

2

-P(j)]

- sin-’

(. - Jz[l

J[l - p ( j ) 1 / 2

-

Fig. 1. Diagrams of the variances for independent pairs of samples ( h = 03)versustheactualvalue of thenormalized acf. Solid lines: (e) full ADA, (b) reduced ADA. Dashedlines:(a) PC, (c) HS, (g) Direct. Dashed/dotted lines: (d) reversed full ADA, (f) double ADA. Simulation results ( 1000 runs): A full ADA, reduced ADA, double ADA.

- P(j)]

+p(j)]]

+ O( 1 / N 2 ) .

(26)

These forms are independent of the actual shape of the acf, and they will be used for comparison purposesin the next section. IV . DISCUSSION From (25) and (26), we observethat the behavior of the ADA estimators is not symmetric with respect to the sign of the estimated normalized acf. In fact, these estimators are especially designed to work in a high correlation environment, i.e., close to the value p ( j ) = 1 . To expand this point, let us analyze the accuracy loss with respect to the Direct method of the proposed estimators and the HS and PC estimators for the case of independent pairs of samples.just near p ( j ) = 1. Using (25) and (26) and the results of [3] and [ 5 ] , we obtain

+

(27)

lirn h-+m P-1

lirn h-03 P-1

var { J ~ J ) ] var

{a&))

=

{P d j ) } var { bD(j)}

var

0.94

This is accomplishedby simply replacing Pj with the correspondent quantity

. N-1

and by changing the signs of the estimates (13) and (14). To take full advantage of this behavior, the estimators could be usedin an adaptive manner if the correlation polarity is known or if it can be deduced from preceeding estimates. In Fig. 1 , the variances of the considered estimators are plotted in a comprehensive diagram for the case of independent pairs of samples. We observe that the variance of the full ADA estimate remains quite low if it is used for negative correlation. In addition, consider that it is boundedby the values, 1 and - 1 . This is not true forthe reduced ADA estimate which is only bounded by 1. On the other hand, its use should be limited just to this zone. Due to the interesting overall behavior of the full ADA estimator, a third ADA estimate can be devised by averaging out the normalandthe reversed ADA full estimates. Thus, a “double” ADA estimator is defined as follows:

+

+

(28) 2.22

w’

(29)

It turns out that, unlike the other fast techniques, the ADA estimators approach the variance of the Direct one. On the other hand, the ADA estimators can be “reversed” to obtain the same behavior for high negative correlation.

This estimate requires 2 N more sums than a A D A ( j ). The evaluation of its accuracy is omitted here for the sake of brevity, but the variance is plotted in Fig. 1 for independent pairs of samples. This estimator is symmetric and is bounded by 1 and - 1 . In general, it appears as thebest

+

‘The quantity ( Pj / eo)’can actually assume values greater than2.

658

IEEETRANSACTIONSONACOUSTI(

:s,

SPEECH, AND SIGNALPROCESSING,

.o Fig. 2. Diagrams of thevariancesfork = 1andfor p ( j ) = exp ( - I j 1 / 8 ) versus the actual values of the normalized acf. Solid lines: full ADA (lower), reduced ADA (upper). Dashed lines: Direct (lower), HS (upper). Dashed/dotted line: double ADA. Simulation results ( N = 500, 10 000 runs): A full ADA, 0 reduced ADA, H double ADA.

available sum-based estimate for a fixed number of samples. However, its behavior toward the points + 1 and - 1 is not as good as the one of the full and reduced ADA estimators. Let us consider now the case h = 1. For the HS estimator, we make use of the .results of [ 5 ] . Fig. 2 shows some variance plots for N = 500. The normalized acf p ( j ) = exp ( - I j 1 / 8 ) (not strictly time limited) is assumed. For small values of N , simulations indicate that the theoretical results tend to overestimate the variance when p ( j ) < 0.5. It is evident that for h = 1, the differences between the estimators are substantially reduced, even if the hierarchy of the variances is the same as for h = 03. Since we are mainly interested in fast estimation of high values of the acf, let us compare the estimators by displaying their accuracy loss (in terms of variance ratios) with respect to the Direct estimator for different values of h using suitable scales in order to investigate the neighborhood of p ( j ) = + l . From Fig. 3, we observe that forhigh correlation, both the full and the reduced ADA estimates are unconditionally preferable to the HS one as far as the variance conis cerned. The advantage is reduced for small values of h. So far, we have compared the variances. Thebias of all the considered estimates actually becomes negligible with respect to the standard deviation for high N because it is bounded to the order O ( 1/ N ) for memory-limited processes. From the theoretical results reported here (23) and in [ 5 ] ,it has been verified that the bias of the ADA estimates is generally superior to the one of the HS estimate, and it increases for decreasing values of h. In Fig. 4, the limit case h = 1 is shown. Finally, two observations are in order.

.7

.9

VOL. ASSP-35, NO. 5, MAY 1987

.97 ,999 ,997 .99

Fig. 3. Diagrams of the losses of the fast estimators with respect to the Direct estimator (variance ratio) versus the actual value of the normalized acf. The abscissa is linear with -log ( 1 - p ) . The ordinates are in natural values and in decibels. The assumed normalized acf is p ( j ) = exp ( - I j 1 /8). Solid lines: full ADA, k = m (lower); reduced ADA, k = m (upper). Dashed lines: (a) PC, h = m; (b) HS, h = m; (c) HS, k = 3; (d) HS, k = 1. Dashed/dotted line: full ADA, k = 1 . Simulation results ( N = 500, 10 000 runs): A full ADA, k = m; 0 reduced ADA, k = m.

-BlasxN

Fig. 4. Diagrams of the bias versus the actual value of the normalized acf f o r k = 1 and p ( j ) = exp ( - I j I /8). Solid lines: (b) full ADA, (a) reduced ADA. Dashed lines: (c) Direct, (e) HS. Dashed dotted line: (d) double ADA. Simulation results ( N = 500, 10 000 runs): A full ADA, 0 reduced ADA, double ADA.

The fulland the doubleADA estimators are extended in a natural fashion to cross-correlation estimates so that they can be employed in the Burg method instead of the optimum one. Since theirmagnitude is bounded by unity, the stability is assured. The fast estimation techniques are based on the invariance property of the Gaussian processes so that their applicability must be verified when other kinds of processes are encountered. Several simulation tests indicate that the robustness of the ADA estimates to the Gaussian assumption is comparable to the one of the HS estimate. V. CONCLUSIONS Two ADA estimators of the normalized acf, based mainly on additions, have been proposed as low-cost methods (13), (14). Their performances in terms of bias

JACOVITTI TECHNIQUE AND CUSANI:

and covariance have been evaluated for quite general operating situations forGaussian, memory-limited processes (23), (24). For the limit case of independent pairs of samples, the variance is expressed in a closed form (25), (26). The theoretical accuracy of the AMDF estimateused in speech processing techniques has been also evaluated (19), (22). The proposed ADA estimators exhibit a quite remarkable accuracy for high correlation estimates when compared to the other available fast techniques. A third ADA estimate (3 1) does not perform so well for high correlation, but is uniformly more accurate than the HS estimate. Due to their features, theADA estimators are attractive for applications where accurate and fast high correlation estimates are required. Moreover, they should be preferred in general to the HS estimate when the number of the available samples is bounded. APPENDIX ACCURACY OF THE ADA ESTIMATES Let us refer to the following zero-mean rv’s: OF EVALUATION

+

The accuracy of the estimate is now evaluated using the above expansion in an approximate way by neglecting the terms beyond the second order. We obtain, for the bias, n

Ix(i

+j)l

-

Pj - var ( q j ) + 2 = cov ( p j , q j ) . Qj”

cov

{ fiADA(jl), fiADA(j2)}

-

x(i

+j)l}

from which the result of (24) is deduced. To evaluate thedegree of approximation of these expressions, let us consider first the neglected third-order term:

. .

(A0

-

Pi = pj

+ Pj

= qj

+ G’j

Qj

N-IN-1

N-1

+j)l}

*

so that we can pose

N- 1

N- 1

N- 1

[lj

E 7pjqj p j and qj being zero-mean rv’s. Expansion of ;ADA( j ) around the expected values and Qj gives

- - [P,? + 2pjpj

+ p?]

E { t j ( g )tj(h)q j ( l ) ) * ( ~ 7 )

Under the hypothesis of a memory-limited process, the moments inside the above sum factorize in first- and second-order moments as long as the differences between the indexes 1 g - h 1, 1 g - 1 I, I h - 1 I exceed a fixed quantity uj depending on the memory of the process and vanish. The volume of the domain where the factorization does not hold is only linearly increasing with N so that actually the overall contribution of this part of the summation must decrease as 1/N2, provided that the moments are finite and multiplied out by l / N 3 . The same circumstance holds for all of the third-order terms. Let us consider now the fourth-order term:

where

1 2

(A5)

Direct substitution of moments (15), (16) , (1 9) , (20), and (21j gives the result of (23). Using the sameprocedure for the covariance,we obtain

\

- E { JxCi) + Ix(i i = o , 1, . * ‘, N - 1

= 1

D2

THE

e j ( i >= I x ( i ) - x ( i + j ) ( - E ( I x ( i ) q j ( i >= I x ( i ) l

659

FOR HIGH CORRELATION ESTIMATION

pi

N - 1 N-1 N-1

N-1

= --

e! N 4 *

g = O ~ S 1=0 O k=O

E{t j ( g ) tj(h) qj(l) qj(k)}* ( ~ 8 )

When the distance between any two indexes g , h, 1, k exceeds the quantity aj, then the fourfold moments in the sum factorize.The products containing odd-order moments are null, whereas the products of twofold moments vanish as 1/ N 2 . The domain where the factorization does not hold has an ipervolume proportional to N rather than to N 4 so that the terms inside give an overallcontribution vanishing as 1/ N 3 . This is true for all of the fourth-order terms [ 111. Consequently, the term (A8) is bounded to 1/ N 2 and the same occurs for the remaining fourth-order terms.

660

VOL. ASSP-35, NO. 5, MAY 1987

IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, SIGNAL AND PROCESSING,

For the fifth-order terms, the unfactorizable contributions are bounded to 1/N4. Where factorization occurs, the products in one- and fourfold moments vanish, whereas the products in two- and threefold moments give contributions bounded to 1/N3. For the sixth-order terms, the unfactorizable contributions are bounded to 1 / N 5 , whereas the different factorized moments contribute at most as 1/N3 (products of two- and fourfold moments). Proceeding by induction, we see that the terms of order 2m and 2 m 1 are both bounded to 1 / N“ due to the hypothesis made of the finite extension of the memory of the process. The same expressions and the same approximation apply to the reduced ADA estimate, provided that Q j , Qj,, Qj2 and qj, qj,, qj2 are replaced with Qo and qo.

+

[lo] R. Cusani and A. Neri, “A modified hybrid sign estimator for the normalizedautocorrelationfunctionofaGaussianstationaryprocess,” IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP33, pp. 1321-1324, Oct. 1985. New York: McGraw-Hill,1977,p. [ l l ] A.Papoulis, SignalAnalysis. 376. [121 J . L. Doob, Stochastic Processes. New York: Wiley, 1953, p. 493.

> ;

>,,,

::

GiovanniJacovittiwasborninAvezzano(AQ), Italy, on September 24, 1945. He received the Dr. Ing.degreeinelectronicengineeringfromthe University of Rome, Rome, Italy, in 1970. In 1970 he joined the Department of Electrical Communications, University of Rome, as an Assistant Teacher. From 1977 to 1979he was an Assistant Professor of Communication Theory at the University of Cagliari. Since 1979 he has been an REFERENCES Associate Professor of Digital Signal Processing at the University of Rome. From 1984 to 1986 he [11 M. J. Ross et al., “Average magnitude difference function pitch exwas also teaching in the University of Bari. He began his research in the tractor,” IEEE Trans. Acoust., Speech, Signal Processing,vol. ASSPfield of data communications through noisy channels. Since 1978 he has 22, pp. 353-362, Oct. 1974. been involved in detection and estimation theory problems and in signal [2] D. Hertz, “A fast digital method of estimating the autocorrelation of processing applications. His main technical works are in the areas of sura Gaussian stationary process,” IEEE Trans. Acoust., Speech, Signal veillance radar, geophysical prospecting, and ultrasonic echography. His Processing, vol. ASSP-30, p. 329, Apr. 1982. [3] K. J. Gabriel, “Comparison of three correlation coefficient estimators theoretical works are currently directed towards spectrum analysis and image recognition problems. for Gaussianstationaryprocesses,” IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-31, pp. 1023-1025, Aug. 1983. 141 G . Jacovitti, A. Neri, and R. Cusani, “On a fast digital method of estimating the autocorrelationof a Gaussian stationary process,” IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-32, pp. 968976, Oct. 1984. Roberto Cusani (“81) was born in Rome, Italy,’ [5] G . Jacovitti and R. Cusani, “Performance3 of the hybrid-sign correon September 29, 1957. He received the Dr. Ing. lation coefficient estimator for Gaussian stationary processes,” IEEE degree in electronic engineering from the UniverTrans. Acoust., Speech, Signal Processing, vol. ASSP-33, pp. 731sity of Rome “La Sapienza” in 1981, and is cur733, June 1985. rently completing the Ph.D. degree at the same [6] S . Cacopardi, “Applicability of the relay correlator to radar signal University. Image coding is the subject of his dis1, 1983. processing,” Electron. Lett., vol. 19, pp. 722-723, Sept. sertation. [7] H.Brehm and W. Stammler, “Application of the relay and polarityIn 1982 he worked at the Communication Recorrelation to the analysis of speech signals,” in Signal Processing search Centre FUB, Rome. During 1985 he H.W.Schussler,Ed.EURASIP, 11: TheoriesandApplications, workedattheEcolePolytechniqueFederale de 1983. Lausanne, Switzerland. Since 1986 he has been a [8] T. Koh and E. J . Powers, “Efficient maximum entropy spectral esat theUniversity of Rome “Tor Vertimation using non-multiplication methods,” in Proc. ICASSP, 1984, Teaching and Research Assistant gata.” His main research interests concern transmission and processing of p. 13.2.1. signals and images, estimation of signal parameters, spectral. estimation, [9]P.C.Egau,“Correlationsystemsinradioastronomyandrelated and image pattern recognition. fields,” Proc. IEE, vol. 131, part F, N.l, Feb. 1984. ; ;

*



, ,,

,,

,:’