A Robust Time-Varying Identification Algorithm Using Basis Functions

Annals of Biomedical Engineering, Vol. 31, pp. 840–853, 2003 Printed in the USA. All rights reserved.

0090-6964/2003/31共7兲/840/14/$20.00 Copyright © 2003 Biomedical Engineering Society

A Robust Time-Varying Identification Algorithm Using Basis Functions RUI ZOU, HENGLIANG WANG, and KI H. CHON Department of Biomedical Engineering, State University of New York at Stony Brook, Stony Brook, NY (Received 5 November 2002; accepted 12 March 2003)

Abstract—We extend a recently developed time invariant 共TIV兲 model order search criterion named the optimal parameter search algorithm 共OPS兲 for identification of time varying 共TV兲 autoregressive 共AR兲 and autoregressive moving average 共ARMA兲 models. Using the TV algorithm is facilitated by the fact that expanding each TV coefficient onto a finite set of basis sequences permits TV parameters to become TIV. Taking advantage of this TIV feature of expansion parameters exploits the features of the OPS, which has been shown to provide accurate model order selection as well as extraction of only the significant model terms. Another advantage of the new algorithm is its ability to discriminate insignificant basis sequences thereby reducing the number of expansion parameters to be estimated. Due to these features, the resulting algorithm can accurately estimate TV AR or ARMA models and determine their orders. Indeed, comparison via computer simulations of AR models between the proposed method and one of the wellknown iterative methods, recursive least squares, shows the greater capability of the new method to track TV parameters. Furthermore, application of the new method to experimentally obtained renal blood flow signals shows that the new method provides higher-resolution time-varying spectral capability than does the short-time Fourier transform 共STFT兲, concomitant with fewer spurious frequency peaks than obtained with the STFT spectrogram. © 2003 Biomedical Engineering Society. 关DOI: 10.1114/1.1584683兴

tion of signals is possible with a parametric model. TV parametric models include autoregressive moving average 共ARMA兲 models and its subclasses: autoregressive 共AR兲 and moving average 共MA兲 models. Due to the compactness and higher resolution afforded by TV parametric methods, their application to physiological systems is diverse, ranging from electroencephalogram2 to electrocardiogram3 signals. Two of the most popular approaches to identifying a TV system, least-mean-square 共LMS兲 and recursive least-squares 共RLS兲, are based on adaptive filters to track the TV coefficients. These methods work reasonably well if the time variations are slow. However, if the TV coefficients change quickly, these adaptive approaches are not able to track the time-varying properties of the system, because the speed of convergence of the algorithms is not fast enough. Another approach to tracking a slowly varying TV system is to employ an ensemble averaging technique. This technique assumes that dynamics of the system do not change significantly as multiple data recordings are obtained. Consistent data recording from either the same or different preparations over prolonged time is often impractical in practice, thus, these conditions preclude this from being a widely utilized approach. A more promising approach, on which we focus, is to expand each TV coefficient onto a set of basis functions. A key advantage of using basis functions is that a considerable reduction in the number of parameters needed to track each TV coefficient can be obtained. Furthermore, identification of fast-varying nonstationary processes can be more properly handled by the use of basis functions than by the RLS approach. However, it should be noted that, as with the RLS, basis function expansion approaches cannot track dynamics that have short-lived ‘‘spike-like’’ behavior. Several basis functions have been proposed, including Fourier sequences,9 Legendre polynomials, Walsh functions, discrete prolate spheroidal sequences 共DPSS兲, and wavelets.13 Two of these basis functions, Legendre and Walsh functions, which we utilize in this article, are described in the Appendix. DPSS

Keywords—Time-varying, Recursive least-squares, Short time Fourier transform, Least-mean-square, Basis functions, Renal blood flow, Time-varying spectrum, Optimal parameter search, ARMA, AR.

INTRODUCTION Most biological signals, no matter how short the data record duration may be, are not stationary, due to the inherent time-varying 共TV兲 characteristics of biological systems. Consequently, a multitude of TV methods have been developed.6,9,14 Time-frequency representation and TV spectral analysis represent nonparametric approaches to modeling nonstationary data. However, as with timeinvariant 共TIV兲 systems, more parsimonious representaAddress correspondence to Ki H. Chon, Department of Biomedical Engineering, SUNY at Stony Brook, HSC T18, Rm. 030, Stony Brook, NY 11794-8181. Electronic mail: [email protected]

840

A New Algorithm for Time-Varying Spectrum Estimation

have similar characteristics as Legendre polynomials, thus were not used here. Application of these basis functions to biological systems has been limited, however, mainly due to difficulty in choosing the appropriate basis function. It is of paramount importance to select the proper basis function because it controls the smoothness and variation speed of TV coefficients. All of the aforementioned parametric TV methods, including the use of basis functions, face the unavoidable task of a priori selection of the system’s model order. The accuracy of tracking a TV system is predicted on determining accurate model order, which is also the case for time-invariant systems. Two popular criteria for determining model order are Akaike’s information criterion 共AIC兲1 and the minimum description length 共MDL兲,10 which are based on asymptotic results and were originally created for TIV systems. These model order criteria have been applied to TV systems, but with limited success. Recent methods have been shown to be more accurate in selecting model order and consequently result in more accurate parameter estimation.7,8 Two of the more recent robust model order selection criteria are the fast orthogonal search 共FOS兲7 and the optimal parameter search 共OPS兲.8 The purpose of this article is to extend the OPS, originally designed for TIV–ARMA models,8 to be applicable to TV systems. The OPS has been shown to be able to obtain a correct model order despite an a priori overdetermined model order, and has the capability to extract only the significant model parameters.8 In almost all cases, the OPS has been shown to provide performance superior to the AIC, and in most cases is more accurate than the FOS. We propose that the TV algorithm of OPS provides not only an appropriate model order selection criterion for TV system identification but also an approach to selecting optimal basis sequences through eliminating the insignificant model terms as well as the associated basis function terms. Judicious selection of model order is interlinked with judicious selection of basis functions. Without accurate model order, the optimal basis sequences are meaningless; without optimal basis sequences, it is difficult to select the correct model order. The Methods section demonstrates this symbiotic relationship. Several computer simulation results are provided to demonstrate the validity of the proposed method.

METHODS TV–AR and ARMA Model Representation via TV–OPS Algorithm A TV–ARMA process is represented by the following equation:

841

P

y 共 n 兲⫽

Q

兺 a 共 i,n 兲 y 共 n⫺i 兲 ⫹ j⫽0 兺 b 共 j,n 兲 x 共 n⫺ j 兲 ⫹e 共 n 兲 ,

i⫽1

共1兲 where a(i,n) and b( j,n) are the time-varying AR and MA coefficients to be determined, respectively, and are functions of time. The indices n is time, and the indices P and Q are time instant, the maximum model orders of the AR and MA models, respectively. We assume that the maximum model orders are TIV. The term e(n) is the prediction error. We expand the TV coefficients a(i,n) and b( j,n) onto a set of basis function ␲ (n): V

a 共 i,n 兲 ⫽

兺

k⫽0

␣ 共 i,k 兲 ␲ k 共 n 兲 , 共2兲

V

b 共 j,n 兲 ⫽

兺

k⫽0

␤ 共 j,k 兲 ␲ k 共 n 兲 ,

where ␣ (i,k) and ␤ (i,k) represent the expansion parameters with V as the maximum number of basis sequences. Setting ␲ (n)⬅1 accounts for the fact that the stationary portion of the model is included, because the basis function ␲ 0 has a constant value for all time n. Substituting Eq. 共2兲 into Eq. 共1兲, we obtain the following: P

y 共 n 兲⫽

V

兺兺

i⫽1 k⫽0 Q

⫹

␣ 共 i,k 兲 ␲ k 共 n 兲 y 共 n⫺i 兲 V

兺兺

j⫽0 k⫽0

␤ 共 j,k 兲 ␲ k 共 n 兲 x 共 n⫺ j 兲 ⫹e 共 n 兲 .

共3兲

The next step is to select proper basis functions so that the TV coefficients can be estimated. We have experimented with three different types of basis functions: Legendre polynomials, discrete prolate spheroidal sequences 共DPSS兲, and Walsh functions.13 Our experience shows that different basis functions show their own unique tractability and accuracy. It is not a requirement for basis functions of the TV–OPS method to be orthogonal or orthonormal. Figure 1 shows characteristics of Legendre and Walsh basis functions. DPSS are not shown because they have similar characteristics to Legendre polynomials. As shown in Fig. 1, Legendre polynomials and DPSS perform well if the coefficients are smoothly changing with time, e.g., sinusoids. Walsh functions on the other hand, behave well for piecewise stationary TV coefficients. The required minimum number of Walsh basis functions to capture necessary dynamics is much greater than either Legendre or DPSS basis functions, however. In summary, Walsh functions should be used when the dynam-ics are expected to exhibit fast transients and

842

ZOU, WANG, and CHON

y k 共 n⫺i 兲 ⫽ ␲ k 共 n 兲 y 共 n⫺i 兲 , 共4兲 x k 共 n⫺ j 兲 ⫽ ␲ k 共 n 兲 x 共 n⫺ j 兲 . Substituting Eq. 共4兲 into Eq. 共3兲 results in the following expression: P

y 共 n 兲⫽

V

兺兺

i⫽1 k⫽0 Q

⫹

␣ 共 i,k 兲 y k 共 n⫺i 兲 V

兺兺

j⫽0 k⫽0

␤ 共 j,k 兲 x k 共 n⫺ j 兲 ⫹e 共 n 兲 .

共5兲

Equation 共5兲 shows that the TV–ARMA model can now be considered to be a TIV–ARMA model, since ␣ (i,k) and ␤ ( j,k) are not functions of time. Thus, the task simplifies to solving for parameters ␣ (i,k) and ␤ ( j,k), which can be more effectively estimated using the OPS algorithm. The algorithm of TV–OPS follows a similar two-step procedure to OPS. The first step of the OPS is to select the linearly independent vectors from the pool of candidate vectors, which is represented by a matrix M representing all candidate vectors y(n⫺1), . . . ,y(n ⫺P),x(n), . . . ,x(n⫺Q), arranged in the following form: M⫽ 关 y0 共 n⫺1 兲 , . . . ,yV 共 n⫺1 兲 ,x0 共 n兲 , . . . ,xV 共 n兲 ,y0 共 n ⫺2 兲 , . . . yV 共 n⫺2 兲 , . . . 兴 FIGURE 1. Dynamic characteristics of the first five orders of Legendre „top panel… and Walsh „bottom… functions. For the top panel the following symbols were used: Solid: first order, dotted: second order, dash–dotted: third order, dash–dash: fourth order, asterisk: fifth order.

共6兲

with y k 共 n⫺i 兲 ⫽ ␲ k 共 n 兲 y 共 n⫺i 兲 ⫽ 关 ␲ k 共 1 兲 y 共 1⫺i 兲 , ␲ k 共 2 兲 y 共 2⫺i 兲 , . . . , ␲ k 共 N 兲

burst-like dynamics, whereas Legendre polynomials are more appropriate for smoothly changing dynamics. Most physiological signals inevitably exhibit both fast and slowly varying dynamics at different stages of the signal, therefore, it might be more appropriate to utilize multiple sets of basis function, i.e., combinations of both Legendre and Walsh basis functions. Furthermore, having multiple sets of basis functions alleviates the problem of determining a proper basis function a priori. We have developed such a technique and incorporated it into the TV–OPS, and it has been shown to be more accurate than using a single set of basis functions.5 Once proper basis functions have been chosen, we define new variables such that:

⫻y 共 N⫺i 兲兴 T , x k 共 n⫺i 兲 ⫽ ␲ k 共 n 兲 x 共 n⫺i 兲 ⫽ 关 ␲ k 共 1 兲 x 共 1⫺ j 兲 , ␲ k 共 2 兲 x 共 2⫺ j 兲 , . . . , ␲ k 共 N 兲 ⫻x 共 N⫺ j 兲兴 T , k⫽1, . . . ,V;

i⫽1, . . . , P;

j⫽0, . . . ,Q;

n⫽1, . . . N, where N is the total number of data points. The pool of candidate vectors results from a priori selection of TV


ARMA model orders P and Q. Note that bold and unbold letters represent vectors and scalars, respectively. The first candidate vector y0 (n⫺1) and the second candidate vector y1 (n⫺1) are used to determine whether they are linearly independent, e.g., use y0 (n⫺1) to fit y1 (n⫺1) by the least-square analysis and calculate the residual of the fit. If the residual is larger than a preset threshold 共for a noiseless signal, the threshold is set to zero兲, then y1 (n⫺1) is considered to be independent from y0 (n⫺1). If not, discard y1 (n⫺1) and select the third candidate vector y2 (n⫺1) and repeat the procedure. Once y1(n⫺1) is determined to be linearly independent from y0 (n⫺1), these two linearly independent vectors are used consecutively to estimate the linear independent of a third vector yi⫹1(n⫺1). Among the pool of P⫻(V⫹1)⫹(Q⫹1)⫻(V⫹1) candidate vectors, this procedure is continued until all of the linearly independent candidate vectors are selected to form a new matrix pool:

843

FIGURE 2. Projection distance difference in percentage vs. model terms.

W⫽ 共 w1 ,w2 , . . . ,wS 兲 , where S is the number of linearly independent candidate vectors. Thus, S is the maximum model order. This sequence of steps provides a new approach to determining a model order. Until now, the popular approaches were to use either AIC or MDL criterion to determine the model order of AR or ARMA processes. With the new matrix of linearly independent vectors, least-squares analysis is performed: y 共 n 兲 ⫽⌰TW⫹e 共 n 兲 ,

共7兲

where ⌰⫽ 关 ␪ 1 , ␪ 2 , . . . , ␪ S 兴 . In Eq. 共7兲, ␪ i is the coefficient estimate of the ARMA model. The objective is to minimize the equation error, e(n), in the least-squares sense using the criterion function defined as follows: N

J N共 ⌰ 兲 ⫽

兺

n⫽1

关 y 共 n 兲 ⫺⌰ T W 兴 2 .

共8兲

The criterion function in Eq. 共8兲 is quadratic in ⌰, and can be minimized analytically with respect to ⌰, yielding the following well-known equation: & ⫽ 关 WW T 兴 ⫺1 Wy. ⌰

共9兲

The second step of the OPS is to determine only the significant model terms among the pool of candidate model terms. The purpose of this task is to identify only the significant model terms among the selected linearly independent vectors. In other words, this allows the ca-

pability to discriminate possible insignificant terms among the many model terms. Note that both AIC and MDL only provide information as to what the maximum model order should be and no clues as to whether or not there are any insignificant model terms. To determine which of the candidate vectors are significant, we calculate the following: C m⫽

1 N

N

兺

n⫽1

&␪ 2 w 共 n 兲 2 , m m

m⫽1, . . . ,S,

共10兲

where S represents the number of linearly independent candidate vectors and ␪ represents the coefficient obtained from Eq. 共9兲. Note that the vector w m is the inner product between basis functions with the input or output vectors as shown below: w m 共 n 兲 ⫽ ␲ k 共 n 兲 y 共 n⫺i 兲 k⫽0, . . . ,V;

or

w n 共 n 兲 ⫽ ␲ k 共 n 兲 x 共 n⫺ j 兲 ,

i⫽1, . . . , P;

n⫽1, . . . ,N;

j⫽0, . . . ,Q;

共11兲

m⫽1, . . . ,S.

Note that the operation of Eq. 共10兲 involves adding all the w m together with the same y(n⫺i) or x(n⫺i) but with different ␲ k . The magnitude values of each C m , as estimated from Eq. 共10兲, are then arranged in descending order. We only retain w m terms that significantly reduce the estimation residuals. This operation has the added benefit of also discarding insignificant basis sequences, ␲ k , at the same time. An approach that can be adopted

844

ZOU, WANG, and CHON TABLE 1. TV–OPS model order selection criterion with the Walsh basis function. The number of basis sequences, V, equals 28. Bold terms are the selected model terms. Projection distance Model terms

325.30

205.83

31.02

28.42

17.67

7.80

y ( n ⫺1)

y ( n ⫺2)

y ( n ⫺4)

y ( n ⫺3)

y ( n ⫺5)

y ( n ⫺6)

to determine the significance of C m terms is to calculate 关 (C m ⫺C m⫹1 )/C m 兴 , convert it to % 共multiple by 100兲, and plot it as a function of m, as shown in Fig. 2. Note that the significant term is characterized by the first maximum value shown in Fig. 2, suggesting that only the first two terms should be used and the remaining terms should be discarded. The rest of the terms are considered to be insignificant because the drop in the normalized projection distance value from the third to the fourth term is small. Therefore, our approach is to use the first maximum value in the plot as the number of significant terms. This figure is based on the projection distance values presented in Table 1, represented in percentages. The only decision to be made is the preset threshold for linear independence search. For clean signals, the linear independency can always be obtained with a threshold setting of 0.0001. For noisecontaminated signals, the threshold should be adjusted to a value higher than 0.0001. It is unclear at this point how the threshold value should be adjusted to compensate for noise since we know neither the signal-to-noise level nor whether the noise is white or colored. A threshold value should only be set when there is a clear knowledge of the noise characteristics, which is normally not available in practice. Therefore, we normally set the threshold value to 0.0001 regardless of whether or not the signal is contaminated by noise. We have shown that this threshold value is not so critical to obtaining a correct model order in time-invariant cases, as the method is robust even up to 0 and 3.5 dB for additive and dynamic noise contamination, respectively.8 The final procedure of the algorithm is to estimate ARMA model terms ␣ (i,k) and ␤ ( j,k) of Eq. 共3兲 using the least-squares method and the calculation of TV coefficients a(i,n) and b( j,n) using Eq. 共2兲. The number of coefficients estimated depends on the choice of TV–ARMA model orders P and Q as well as the selection of V in Eq. 共3兲. Specifically, the total number of coefficients to be estimated is: P(V⫹1)⫹(Q ⫹1)(V⫹1). Note that the RLS utilizes far more parameters than does our approach because many parameters are used to characterize a single datapoint, which could result in an exorbitant number of parameters over the duration of the data. The TV–OPS, however, uses a fixed number of parameters P(V⫹1)⫹(Q⫹1)(V⫹1) for all datapoints. Consequently, a significant saving in the

number of parameters used to characterize time-varying dynamics is obtained with the TV–OPS. SIMULATION EXAMPLES In this section, we demonstrate the efficacy of the TV–OPS algorithm with several simulation examples. For all simulation examples to be considered here, an incorrect model order of 6 AR is selected for an AR process, and 6 AR and 5 MA are selected as the incorrect model orders for an ARMA process. Example 1 The first simulation example considers a 600 data point second-order TV–AR model. This model is described by the following difference equation: y 共 n 兲 ⫽a 共 1,n 兲 y 共 n⫺1 兲 ⫹a 共 2,n 兲 y 共 n⫺2 兲 ⫹e 共 n 兲 , where e(n) is zero-mean Gaussian white noise 共GWN兲 with a unit variance. The first coefficient term is designed to be TV and is described by the following: a 共 1,n 兲 ⫽2 cos关 2 ␲ f 共 n 兲兴 ,

f 共 n 兲⫽

再

0.4 n⫽1 – 200 0.2 n⫽201– 400, 0.4 n⫽401– 600

and the second coefficient has constant value for all times such that: a 共 2,n 兲 ⫽⫺1,

n⫽1 – 600.

We compare the TV–OPS method to recursive least squares 共RLS兲 with variable forgetting factors 共FF兲, developed by Cho et al.4 RLS is chosen because it has been shown to achieve faster rates of convergence with smaller excess mean-square errors than least mean square 共LMS兲, and it is widely utilized in practice. For this simulation example, we expanded TV coefficients onto Walsh functions with 28 basis sequences. We choose Walsh basis functions because they have been shown to provide good approximation for piecewise sta-


845

TABLE 2. Mean square error as a function of M „number of basis functions….

M MSE

4 44.21

8 29.17

12 22.72

16 17.92

20 15.76

24 14.43

28 10.75

32 182.03

36 266.33

corrupted by the dynamic noise source e(n), TV–OPS is able to determine the correct AR model order of 2. It should be noted that model order selection is not so sensitive to the proper choice of the number of basis functions. For all choices of the number of basis functions we have tested in Table 2, the TV–OPS provided the correct model order of 2. Figure 3 shows the estimation of model order using minimum description length 共MDL兲. Clearly, Fig. 3 suggests a model order of greater than or equal to 4 should be chosen rather than the true model order of 2. The top panel of Fig. 4 shows the estimated TV coefficients 共dotted line兲 using TV–OPS and the bottom panel of Fig. 4 shows RLS with variable FF 共dashed–dotted line兲 along with true coefficients 共solid line兲. Not surprisingly, since it was given the wrong model order by MDL, the performance of RLS with variable FF degrades. The coefficient a(2,n) obtained by the TV–OPS deviates from the true coefficient, especially at time points 200 and 400 s. However, these deviations are several magnitudes less than those of the RLS. It should be noted that if the model order is correctly determined, then RLS with variable FF provides good performance because the example involves slowly

tionary coefficients, as is the case with this example. A criterion we have adopted for determining the proper number of basis functions is based on calculation of mean-square error 共MSE兲. Table 2 shows the MSE values as the number of Walsh basis sequences was varied from 4 to 36. We note the gradual decrease in MSE values as the number of basis sequences was increased, with the minimum value occurring at 28 Walsh functions; the MSE values significantly jumped to higher values beyond 28 basis functions. Each MSE value, shown in Table 2, was calculated after determining the proper model order as detailed in the following paragraph. Table 1 shows the model order selection criterion of TV–OPS based on the selection of 28 Walsh functions. The projection distance in Tables 1, 3, and 4 refers to the distance of a projection from one vector to another linearly independent vector, and is calculated using Eq. 共13兲. The first two terms y(n⫺1) and y(n⫺2) are more significant than the other terms, as evidenced by the higher projection distance value shown above each model term. Despite a priori incorrect model order selection of six AR terms, in addition to having the output

TABLE 3. TV–OPS model order selection criterion with Legendre polynomials. The number of basis sequences, V, equals 8. Bold terms are the selected model terms. Clean signal Projection distance Model terms

0.1948

0.1459

0.0951

0.0208

0.0161

0.0102

y ( n ⫺4)

y ( n ⫺1)

y ( n ⫺2)

y ( n ⫺5)

y ( n ⫺3)

y ( n ⫺6)

0.1822

0.1576

0.1151

0.0263

0.0206

0.0095

y ( n ⫺4)

y ( n ⫺1)

y ( n ⫺2)

y ( n ⫺5)

y ( n ⫺3)

y ( n ⫺6)

20 dB noise Projection distance Model terms

TABLE 4. TV–OPS model order selection criterion with Legendre polynomials. The number of basis sequences, V, equals 8. Bold terms are the selected model terms. Clean signal Projection distance Model terms

3.0398 y ( n ⫺1)

2.5271 y ( n ⫺2)

0.4644 x(n)

0.0902 x ( n ⫺1)

0 x ( n ⫺2)

0 y ( n ⫺5)

0 x ( n ⫺5)

0 x ( n ⫺3)

0 y ( n ⫺4)

0 x ( n ⫺4)

0 y ( n ⫺6)

20 dB noise Projection distance Model terms

2.5359

2.1375

0.4753

0.1057

0.0047

0.0038

0.0037

0.0036

0.0036

0.0036

0.0032

0.0017

y ( n ⫺1)

y ( n ⫺2)

x(n)

x ( n ⫺1)

x ( n ⫺4)

y ( n ⫺6)

y ( n ⫺4)

y ( n ⫺3)

x ( n ⫺2)

x ( n ⫺5)

x ( n ⫺3)

y ( n ⫺5)

846

ZOU, WANG, and CHON

time-varying coefficients. The performance of RLS would have been worse had we chosen a faster TV system. This is in agreement with findings in the literature that RLS cannot track fast time-varying dynamics.2,13 To further demonstrate the superior performance of TV–OPS over RLS, the next example considers faster variations in the AR coefficients. Example 2 Consider a fourth-order TV–AR model with a missing term 关 y(n⫺3) 兴 , with e(n) as a zero-mean 512 data point GWN with unit variance: y 共 n 兲 ⫽a 共 1,n 兲 y 共 n⫺1 兲 ⫹a 共 2,n 兲 y 共 n⫺2 兲 ⫹a 共 4,n 兲 y 共 n⫺4 兲 ⫹e 共 n 兲 . FIGURE 3. MDL values vs. model orders.

The TV coefficients a(1,n) and a(4,n) are sine waves, a(2,n) is constant, and the coefficient a(3,n) is missing

FIGURE 4. Estimation of TV coefficients a „1,n … and a „2,n … using TV–OPS „top panel… and RLS with variable FF „bottom panel…. Solid and dashed lines represent the true and the estimated coefficients, respectively.


847

FIGURE 5. MDL value used to estimate model order for clean „left panel… and noise corrupted „20 dB… signal „right panel….

from the above equation. We use eight-term Legendre polynomials as the basis functions because this selection results in a low MSE value as compared to other choices of number of basis functions. In addition to dynamic noise e(n) in the above equation, GWN which is independent from e(n) was added so that the signal-to-noise ratio 共SNR兲 is 20 dB. Table 3 shows the model order selection criterion of TV–OPS for both clean and 20 dB noise-corrupted signals. Note that for TV–OPS, the selection of model order is equivalent to determining the significant model terms, and the missing term, y(n ⫺3), is readily discriminated for both noiseless and noise-contaminated signals. We observe that there is a significant drop in projection distance value after the first three terms for both noiseless and noise-corrupted signals. Figure 5 shows model order selection using MDL for both noiseless and noisy signals. Even for a clean signal, MDL is not able to provide accurate model order selection as Fig. 5 suggests that the model order should be as great as 5. Furthermore, MDL only provides information regarding the maximum model order and no indication as to whether or not there are any insignificant terms among the chosen model order terms. For 20 dB noise, MDL selection degrades further. Using the model orders selected by TV–OPS and MDL for the 20 dB additive noise signals, Fig. 6 compares the performance of TV coefficient estimation for the two methods. TV– OPS is able to track both time-varying and constant coefficients. To the contrary, RLS fairs rather poorly in tracking the coefficients. It appears that RLS tracks around mean value of the coefficients. The coefficient a(2,n) obtained by the TV–OPS deviates more so than

the other coefficients from the true value, but it is far more accurate than a(2,n) obtained by RLS. To examine the performance of TV–AR spectra, we show in Fig. 7 TV spectra obtained by the TV–OPS 共top right panel兲, short-time Fourier transform 共bottom left panel兲 and RLS 共bottom right panel兲, along with the true spectrum 共top left panel兲 obtained from Example 2. Clearly, the TV–AR spectrum estimated by the TV–OPS provides the closest resemblance to the true spectrum. Both RLS and STFT provided overall four main peaks in the spectrum but the quality of both time and frequency resolution of these two methods are not comparable to the TV–OPS. The STFT spectrum also tends to show additional erroneous frequency peaks. This example demonstrates that accuracy of parameter estimates is important in obtaining accurate TV spectrum, as we have observed in Fig. 6, which shows that TV–OPS provides more accurate tracking capability than does RLS. Example 3 The previous two examples considered TV–AR models. In this example, we consider a TV–ARMA model with model orders of AR⫽2 and MA⫽1, provided below: y 共 n 兲 ⫽a 共 1,n 兲 y 共 n⫺1 兲 ⫹a 共 2,n 兲 y 共 n⫺2 兲 ⫹b 共 1,n 兲 x 共 n 兲 ⫹b 共 2,n 兲 x 共 n⫺1 兲 . The input, x(n), is 512 data point GWN. The TV coefficients a(1,n) and b(2,n) are linearly increased and

848

ZOU, WANG, and CHON

FIGURE 6. „a… Estimation of TV coefficients using TV–OPS „dotted line…. Solid lines: true values, dashed lines: estimated values: 20 dB noise-corrupted case.

decreased, respectively. The coefficient a(2,n) is constant and b(1,n) is a sine wave. We expand the TV coefficients using eight-term Legendre polynomials and consider both noiseless and 20 dB additive noise signals. As described in Simulation Ex-

ample 1, we chose 8 Legendre polynomials because it provided one of the lowest MSE values as we varied the number of basis functions from 4 to 40. Similar to previous examples, we found a significant increase in MSE value as the number of basis functions increased from 20


849

FIGURE 7. Comparison of TV AR spectra of TV–OPS „top right panel…, STFT „bottom left panel…, and RLS „bottom right panel… to the exact spectrum „top left panel….

共MSE⫽1.07e⫺27兲 to 22 共MSE⫽0.45兲. In addition, we found that the TV–OPS provided consistently correct model orders as we varied the number of basis functions from 4 to 40. This example corroborates our previous examples that the model order determination is insensitive to wide ranges of the number of basis functions. As shown in Table 4, TV–OPS is able to discriminate the true model terms from the insignificant model terms even with 20 dB noise. This is evidenced by the sharp drop off in the projection distance value after the first four

model terms. With the noiseless signal, shown in the left panel of Fig. 8, the TV–OPS tracks the variation of the TV coefficients flawlessly, as evidenced by the overlapping of the true and estimated coefficients. Even with 20 dB noise 共right panel of Fig. 8兲, TV–OPS does a remarkable job in tracking time-varying as well non-timevarying coefficients. To determine how well the TV– OPS tracks the original time series under the 20 dB noise contamination, we plot in Fig. 9 512 s of the original time series 共solid line兲 and the estimated time series

850

ZOU, WANG, and CHON

FIGURE 8. TV coefficients via TV–OPS for noiseless „left panel… and 20 dB noise „right panel… corrupted signals. Note that true and estimated coefficients are overlapped for noiseless signal. Solid lines and dotted lines of the right panel represent true and estimated values, respectively.

共dotted line兲. It is not surprising that the estimated time series tracks the original signal rather accurately, since the estimated coefficients did not deviate significantly from the true coefficients 共shown in Fig. 8兲.

FIGURE 9. Original time series „solid line… vs. estimated „dotted line… signal based on TV–OPS. SNRÄ20 dB.

Application of the TV–OPS to Renal Blood Flow Data We demonstrate the application of the TV–OPS to experimentally obtained single nephron renal blood flow data. Furthermore, we compare the TV–OPS spectra of blood flow signals to those obtained using short-time Fourier transform 共STFT兲 spectra. For the TV–OPS, we used eight Legendre polynomials because this number provided the lowest MSE value. Increasing the number of basis functions beyond eight resulted in significantly higher MSE values. For the RLS TV spectrum, a model order of 8 was used, based on the MDL criterion. For the TV–OPS, the initial model order was set to 10 from which only 6 AR parameters were selected for the TV spectrum. Data Acquisition and Experimental Procedure: Experimental methods are described in detail in Ref. 12 and will be briefly summarized. The experimental data were collected from normotensive Sprague–Dawley rats and spontaneous fluctuations of single nephron blood flow signals were obtained. Spontaneous variation of efferent arteriolar blood flow was measured in individual efferent arterioles by focusing the He–Ne laser 共Doppler velocimetry device兲 spot onto the welling point 共star vessel兲 on the renal surface. The laser Doppler velocimetry used to obtain blood flow measurement was carried out by the use of noninvasive transmit and receive probes. The flow probes were mounted in standard micropipette mounts designed for a micromanipulator, and were separated by an angle of 38°, which was previously found to be the optimal angle to obtain accurate blood flow measurements.13 This technique of mounting the flow


851

FIGURE 10. Representative TV spectra from two separate recordings of renal blood flow signals. Top, middle, and bottom panels represent STFT, RLS, and TV AR spectra, respectively.

probes has been shown to reduce the positional sensitivity of conventional laser-Doppler velocimetry devices. The excitation laser was delivered by the transmit probe 共via fiber optics兲 and was moved linearly on its axis to focus the laser spot on the blood vessel of choice 共without actually touching the surface兲. The scattered laser signal was measured by another optical fiber 共receive

probe兲 which was rotated in-plane and tilted out of plane to bring the optical fiber as close as possible to the laser spot, ensuring that the largest portion of scattered light received by the fiber came directly from the vessel of interest. The resultant received signal was delivered to a laser-Doppler velocimeter to process the estimated Doppler frequency. The blood flow rate obtained via the

852

ZOU, WANG, and CHON

laser-Doppler velocimeter is representative of the number of red blood cells moving through a volume of tissue per unit time, and thus it is not an absolute value, but is statistically derived.11,12 Data was acquired by means of an analog-to-digital converter. The sampling rate was approximately 9 Hz. Subsequently, each of the experimental data records used for analysis was 512 s long, with a sampling rate of 1 Hz, after digital low-pass filtering to avoid aliasing. Figure 10 shows two representative TV spectra of renal blood flow obtained by the TV–OPS 共bottom panels兲, RLS 共middle panel兲, and STFT 共top panels兲. We chose STFT mainly because it does not provide an undesired cross-term frequencies. Therefore, we felt it was an appropriate method to compare to the new method. For the STFT we have chosen to accentuate frequency resolution rather than time resolution. Note that for the STFT, due to one of Heisenberg’s uncertainty principles, choosing finer resolution for the frequency domain compromises time domain resolution. The observed contour peak in the frequency interval of 0.1–0.3 Hz has been attributed to the myogenic mechanism for the single nephron blood flow recordings, and it has been widely recognized as one of the two autoregulatory mechanisms regulating renal blood flow. The second autoregulatory mechanism, tubulomerular feedback 共TGF兲, is known to have autonomous, limit cycle oscillations in the 0.03– 0.05 Hz frequency band. Both TV–OPS and RLS show similar frequency characteristics, overall to those of the STFT. However, since these two model-based approaches are based on a limited number of parameters, it is expected that not all frequency peaks observed in the STFT will be reflected in the spectra of TV–OPS and RLS. However, both autoregulatory mechanisms’ contour peaks are identified in the TV–OPS and RLS spectra as well as the STFT. One notable difference is that the TV spectrum obtained with the TV–OPS clearly provides finer resolution in both time and frequency domain than do the spectra obtained with STFT and RLS. With STFT we observe many clusters of frequency peaks that have no physiological significance. The RLS spectrum shows many similar frequency peaks as that of TV–OPS but at the expense of greater parameters and less time and frequency resolution.

models, is shown to be robust even in the presence of significant noise and is able to discriminate missing terms in the model. It is this feature of accurate selection of model structure that makes it possible to track the TV coefficients where even the sophisticated RLS cannot. As shown in Eq. 共4兲, because basis functions are intertwined with model terms, the model order search criterion of OPS allow the added benefit of eliminating insignificant basis function terms. We are not aware of other algorithms that are able to track both slow- and fast-varying systems as the proposed method. Comparison of the renal blood flow spectra obtained with TV–OPS and STFT clearly demonstrates the higher resolution capability of the TV–OPS than STFT. With an STFT spectrogram, many nonphysiological clusters of frequency peaks were obtained. An added benefit of the TV–OPS is that there is no resolution tradeoff between time and frequency representation, as there is with STFT. Given the accuracy of model order selection and the ability to prune out insignificant model terms among the chosen model orders demonstrated by the TV–OPS, these important features enhance the tractability of the proposed method for calculating TV spectrum. Another potential utility of the proposed approach lies in the fact that it can be effectively used to estimate TV transfer functions. Note that for TV systems, TV transfer functions cannot be obtained by simple division of TV cross spectra by the TV auto spectra, as is done for TIV cases with respective TIV spectra. Furthermore, the proposed algorithm can be easily modified to estimate nonlinear TV transfer functions which may potentially lead to better characterization of the dynamics underlying physiological systems.

ACKNOWLEDGMENT This work was supported by the National Institute of Health 共NIH兲 under Grant No. R01 HL069629.

APPENDIX Legendre polynomials P l (x) can be expressed by Rodrigues’ formula

CONCLUSIONS We presented an algorithm for accurate estimation of TV systems via the use of the OPS. By expanding each TV coefficient onto a finite set of basis functions, we have shown that the model order selection criterion of TV systems becomes equivalent to determining model order for TIV–ARMA. This observation allowed us to use the previously developed OPS for determining accurate model order. The OPS, originally designed for TIV

P l共 x 兲 ⫽

1

dl

2 l l! dx l

共 x 2 ⫺1 兲 l ,

where x is a real number between ⫺1⭐x⭐1 and l is the order of polynomial, l⫽0,1,2,3, . . . . The following illustrates the recurrence relations to generate Legendre polynomials:

A New Algorithm for Time-Varying Spectrum Estimation 2

P 0 共 x 兲 ⫽1, P 1 共 x 兲 ⫽x, 共 l⫹1 兲 P l⫹1 共 x 兲 ⫽ 共 2l⫹1 兲 x P l 共 x 兲 ⫺l P l⫺1 共 x 兲 .

Walsh functions are functions consisting of a number of fixed-amplitude square pulses interposed with zeros. Walsh functions can be generated a number of ways. One is to define Walsh functions as the following: p⫺1

W k共 t 兲 ⫽

兿

r⫽0

sgn共cos k r 2 r ␲ t),

where 0⭐t⭐1 and k is the order, k⫽0,1,2,3, . . . p⫺1

k⫽

兺

r⫽0

k r2 4,

k r ⫽0 or 1.

For example, W 5 (t) has the following form: k⫽5⫽1⫻2 2 ⫹0⫻2 l ⫹1⫻2 0 , k 2 ⫽1,

k 1 ⫽0,

k 0 ⫽1,

W 5 共 t 兲 ⫽sgn共 cos k 2 2 2 ␲ t 兲 •sgn共 cos k 1 2 1 ␲ t 兲 •sgn共 cos k 0 2 0 ␲ t 兲 . REFERENCES 1

Akaike, H. Power spectrum estimation through autoregression model fitting. Ann. Inst. Stat. Math. 21:407– 419, 1969.

853

Arnold, M., W. H. R. Miltner, H. Witte, R. Bauer, and C. Braun. Adaptive AR modeling of nonstationary time series by means of Kalman filtering. IEEE Trans. Biomed. Eng. 45:553–562, 1998. 3 Barbieri, R., A. M. Bianch, J. K. Triedman, L. T. Mainardi, S. Cerutti, and J. Philip Saul. Model dependency of multivariate autoregressive spectral analysis. IEEE Eng. Med. Biol. Mag. 16:74 – 85, 1997. 4 Cho, Y. S., S. B. Kim, and E. J. Powers. Time-varying spectral estimation using AR Models with variable forgetting factors. IEEE Trans. Signal Process. 39:1422–1425, 1991. 5 Chon, K. H., and R. Zou. Multiple time-varying dynamic analyses using multiple sets of basis functions 共submitted兲. 6 Cohen, L. Time-Frequency Analysis. Englewood Cliffs, NJ: Prentice-Hall, 1995. 7 Korenberg, M. J. A robust orthogonal algorithm for system identification and time series analysis. Biol. Cybern. 60:267– 276, 1989. 8 Lu, S., K. H. Ju, and K. H. Chon. A new algorithm for linear and nonlinear ARMA model parameter estimation using affine geometry. IEEE Trans. Biomed. Eng. 48:1116 –1124, 2001. 9 Marmarelis, V. Z. Nonlinear and Nonstationary Modeling of Physiological System: An Overview in Advanced Methods of Physiological System Modeling, edited by V. Z. Marmarelis. Los Angeles: Biomedical Simulations Resource, Univ. Southern California, 1987, Vol. 1, pp. 1–24. 10 Rissanen, J. A universal prior for the integers and estimation by minimum description length. Ann. Stat. 11:417– 431, 1983. 11 Shepherd, A. P., and P. A. Oberg, Laser Doppler Blood Flowmetry. Boston, MA: Academic, 1990. 12 Smedley, G., K. P. Yip, A. Wagner, S. Dubovitsky, and D. J. Marsh. A laser Doppler instrument for in vivo measurements of blood flow in single renal arterioles. IEEE Trans. Biomed. Eng. 40:290–297, 1993. 13 Tsatsanis, M. K., and G. B. Giannakis. Time-varying system identification and model validation using wavelets. IEEE Trans. Signal Process. 41:3512–3523, 1993. 14 Williams, W. J. Reduced interference distributions: Biomedical applications and interpretations. Proc. IEEE 84:1264 – 1280, 1996.