A Fast On-line Generalized Eigendecomposition Algorithm for Time Series Segmentation Yadunandana N. Rao and Jose C. Principe Computational NeuroEngineering Laboratory Department of Electrical and Computer Engineering University of Florida, Gainesville, FL 3261 1 e-mail: yadu @cnel.ufl.edu,
[email protected]
Abstract This paper presents a novel, fast converging on-line rule for Generalized Eigendecomposition (GED) and its application in time series segmentation. We adopt the concepts of deflation and power method to iteratively estimate the generalized eigencomponents. The algorithm is guaranteed to produce stable results. In the second half of the paper, we discuss the application of GED to segment time series. GED is tested for chaotic time series and speech. The simulation results are compared with the venerable Generalized Likelihood Ratio Test (GLR) as a benchmark to gauge performance.
ponents that circumvents the slow convergence of gradient methods. The power method can be used to estimate the principal eigenvector and for the subsequent eigencomponents, a deflation procedure is adopted [lo]. Starting from an initial estimate of the principal eigenvector, the algorithm converges to the principal eigenvector e, of a matrix A, with an exponential rate as O((A,&,f)
iteration index and h, h, are the two largest eigenvalues of the matrix A. The convergence is extremely fast when these eigenvalues are well separated. Even for h, h, the speed of convergence and stability are better than the traditional gradient descent algorithms found in the literature.
-
2.0 Generalized Eigendecomposition
1.0 Introduction Principal Component Analysis (PCA) and Generalized Eigendecompositions are widely used in various signal processing applications like feature extraction, signal estimation and also speech separation [l], [2], [3], [8]. Linear Discriminant Analysis (LDA) is also a generalized eigenvalue problem. There are analytic methods for estimating the generalized eigencomponents [6]. However, for many applications, an adaptive on-line solution is required. Sanger’s rule [9] and APEX [2] present iterative solutions for eigendecompositions. [3] presents an iterative algorithm for LDA using two single layered feedforward networks, each modeled according to Rubner and Tavan [4], [SI.The algorithm exhibits slow convergence when the dimensionalities are high and when multiple eigencomponents need to be estimated. [lo] gives a rule only for the largest generalized eigencomponents. [7] proposes an on-line local algorithm for GED. Although the formulation is novel, this algorithm suffers from the problems of gradient methods. Our technique uses the idea of implementing GED using a two step PCA process similar to [3]. We utilize the well known power method [6] for computing the principal com-
0-7803-5800-7/00/$10.00 02000 IEEE
where k is the
266
The generalized eigenvalue problem can be mathematically expressed as A V = AB V , where A, B are non-singular matrices. For signal processing applications, these are generally full covariance matrices of zero mean, stationary random signals. For real symmetric positive definite covariance matrices, the eigenvectors are all real and the eigenvalues strictly positive. Generalized eigenvectors accomplish simultaneous diagonalization of covariance matrices as ?AV = A, ?BV = I where I is an identity matrix. [7] proposes an alternative formulation to solve the GED problem. Assuming A, B as the estimates of the covariance matrices of two zero mean stationary random signals, the problem is to find vi E Rm that maximizes J(Vi)
=
( vTA vi) subject to viTSvj = 0,S = A, B, (A + B) . ( vTBvi)
Taking the gradient of J ( v i ) , we get
VJ.; (l/(vrBvi))[Avi- ((v~Avi)/(v~Bvi))Bvi]. Equating the gradient to zero, VJ =
o+
~ =v ( (~ v ~ ~ v ~ ) / ( v ~ ~ w v e~ ) ) ~ v ~ .
see that this is nothing but the generalized eigenvalue equation, Avi = hiBvp hi = ( vTi A v i ) / ( vTiB v i ) . hi is the only globally asymptotically stable fixed point and this is the generalized eigenvalue. Based on the above interpretation, we can see that the principal component of A is now steered by the distribution of B . For this reason, GED is also termed as Oriented PCA [lo]. The generalized eigenvectors work as filters in the joint space of the two signals, minimizing the energy of one of the signals and maximizing the energy of the other at the same time. They are notch filters that put nulls at the frequencies where one signal has maximum energy. Thus GED can be used as a filter that can perform signal separation by suppressing the undesired component in a composite signal. This property can be used to quantify changes in the time series. We present a new way of segmenting time series using this idea in the second half of the paper.
2.1 Description of the proposed method Figure 1 shows the block diagram for the proposed method.
t
where xl(n)is the data vector at the nth time instant. The first processing block with a full weight matrix W, implements a whitening transform on the dataset x , ( n ) . So, where VI is the eigenvector matrix and
W, =
A,the corresponding diagonal eigenvalue matrix of the covariance matrix SI.The eigendecomposition of SI is done by the power method followed by the standard deflation procedure [lo]. As the next step, the second dataset x2(n)is passed through the fust block and its outputs are used to train the second block. Thus the second processing block with full weight matrix W,implements an eigendecomposition on the dataset x,(n) after being transformed by the matrix W,. The overall mapping performed is V = W , W, W, = AT1’2V,. This matrix is the generalized eigenvector matrix. The variances of the outputs of the second processing block, when the dataset x2(n) is input, are the eigenvalues in descending order of magnitude. We give a simple proof to substantiate our statement. proof: We prove that the overall mapping V simultaneously diagonalizes the covariance matrices SI,S, .
t W, = V,AT1’2, SIVI = A, VI . For simplicity, the time index
,
is not used in the derivation. The output y (n) is
Figure 1: Block diagram
