Fsn=Fs. N=size(abc_new,2); ns=1;. N_act=round(N/2-1);. // N_act=20; no_y=2;. // ax=ym(:,1:11);N=size(ax,2); c=0; H0=[];. // N_act=20; for i=1:N_act ic=i*ns+1;.
PCA and ERA with Hankel
june 2015
Introduction(The intuition): The PCA helps identify important components in orthonormal eigen basis for symmetrical matrices. So the idea is if we ERA after PCA then the variables are independent.In fact this could perhaps help in reducing the dimension in the outputs of the filterbank. In the following we first consider how to obtain coord when a basis change is made. Next we see the PCA working on an ellipse which is oriented at 45 deg, Finally we apply PCA to Hankel.
1. New Coords when the basis is changed. // Expt Change of basis coords and Eigen vectors /SVD //1. How can we change the coords of a vector for a new basis? Recall in standard basis A the coords vector say in R^2 is v' = [ 2 3] = 2 [1 0 ] +3 [ 0 1] = 2e1' +3e2' . where e1 and e2 are standard basis vectors . Now let us define new linearly independent basis f1 and f2 as f1'= [4 5 ] and f2'= [1 3 ] . We want the coords of v in new basis . As f1= 4* e1 +5*e2 and f2= 1* e1 +3*e2 we can write [ f1 = [4 5 ] * [e1 f2 ] [1 3 ] e2] or as f = P*e where P is the matrix [4 5 ] [1 3 ] So
e=P^-1*f
and with v' = [ 2 3] coords in std basis = 2e1' +3e2' = [ 2 3] *e we get new cords with f1 and f2 basis from [ 2 3] *[P^-1*f ] as
fcords=
[ 2 3] *[P^-1 ] = [ 0.4285714
P^-1 = 0.4285714 - 0.7142857
0.2857143 ]
- 0.1428571
0.5714286
Check :fcords means if [ f1 f2] are the axes then the cords of v are [ 0.4285714 0.2857143 ] which means v = 0.4285714*f1+ 0.2857143*f2 =fcords*f but if we plot with std axis v = fcords *f = fcords *P*e (using f = P*e ) and
fcords *P = std coords = [ 0.4285714 =[ 2.
0.2857143 ]* [4 5 ] [1 3 ]
3. ]
If f1 and f2 are orthonormal (f1'*f2 =dot product = 0 ) then it is easier for further analysis of data. Orthonormal Eigen vectors basis: consider a symmetrical matrix a=[4 5 5 3 ] We can obtain the eigen basis from Scilab 5.3.3 as [eig ,lam] = spec(a) lam = - 1.5249378 0. 0. 8.5249378 eig = 0.6710053 - 0.7414525 - 0.7414525 - 0.6710053
f1=[eig(1:2,1:1)] ;//first eigenvector = 0.6710053 - 0.7414525 and
f2 = [eig(1:2,2:2)] = - 0.7414525 - 0.6710053 So p= [f1 f2] =eig = 0.6710053 - 0.7414525 - 0.7414525 - 0.6710053
Now let us plot the vector point v = [ 2 3] in std basis . plot2d( v(1,1) ,v(1 ,2),-4 );
we can plot the eigen basis f1 and f2 also af =linspace(-4, 4 , 10); af1 = f1*af; af2 = f2*af; figure(1); plot2d(af1(1,:) , af1(2,:) , 2); plot2d(af2(1,:) , af2(2,:) , 2);
Now plot vector point v =[2,3] with
plot2d( v(1,1) ,v(1 ,2),-4 );
champ([0] ,[ 0] ,f2(1,1),f2(2,1),5,[-1,-1,1,1] ); champ([0] ,[ 0] ,f1(1,1),f1(2,1),5,[-1,-1,1,1] );
suppose we want to plot with eigen axis as base fcords= [ 2 3] *[eig ] figure(2); plot2d( fcords(1,1) ,fcords(1 ,2),-4 ); plot2d( v(1,1) ,v(1 ,2),-4 ); plot2d(af1(1,:) , af1(2,:) , 2); plot2d(af2(1,:) , af2(2,:) , 2);
//xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 2. Application on an ellipse. PCA is used to obtain eigen basis whic corresponds to major/ minor axis of oriented ellipse. // ellipse expt 3 // EXPT 3 ellipse. nr=10 //circlee y= sqrt(1-x^2) a=1 b=4 //ellipse y=b*sqrt(1-x^2/a/a) x= linspace (-a,a,nr) ; y= b*sqrt(1-x^2/a/a) yf= [y -y] ; xf =[x x] ; x=xf;y=yf; ellip =[x ; y] ; thet=%pi/4; rot=[cos(thet) sin(thet) ; - sin(thet) cos(thet) ] ;// rotate ellpse elliprot= rot*ellip; subplot(221); plot2d(elliprot(1,:),elliprot(2,:),-4); x= elliprot(1,:) ; y=elliprot(2,:) ; xf=x;yf=y;
//b=a(3:4, 2:5) part of a matrix
xexp=sum(xf)/2/nr // mean of x yexp=sum(yf)/2/nr// mean of y
xe=xexp*ones(1,2*nr) ; // exp vector x ye=yexp*ones(1,2*nr) ; // exp vector y xc1=xf-xe; yc1=yf-ye; subplot(221); plot2d(xf,yf,-2); subplot(222); plot2d(xc1,yc1,-2); //compute covar matrix c(1,1) = (xc1*xc1')/(2*nr-1);// covariance of xc1 with itself c(1,2) = (xc1*yc1')/(2*nr-1);// covariance of xc1 with yc1 c(2,1) = c(1,2) ;//symmetric c(2,2) = (yc1*yc1')/(2*nr-1);// covariance of yc1 with itself [U,S,V]=svd(c) ;// V is eigen matrix U is Vtr S =diaglambda
xplt = linspace(-2,2,10); e1=[V(1:2,1:1)] ;//first eigenvector subplot(222); e1xy=e1*xplt;// eige1 plot plot2d(e1xy(1,:),e1xy(2,:),-3); e2=[V(1:2,2:2)] ;// second eigenvector e2xy=e2*xplt;// eig2 plot subplot(222); plot2d(e2xy(1,:),e2xy(2,:),-4); rda=[xc1' yc1' ] ;//centerd data vectors transposed ndt=U*rda' ;//centered data with eigen vecors as axis (basis) subplot(223); plot2d(ndt(1,:) ,ndt(2,:) ,-2);// plot on eigen basis //y=[d(1:nr,2:2)] ;// the second column of data xred= [ndt(1:1,1:2*nr)] ;//reduced dimension //in eigen basis by taking first row of ndt back= V(1:2,1:1)*xred ; xb=back(1:1,1:2*nr) ; yb=back(2:2,1:2*nr) ; plot2d(xb,yb,-2); xbe= xb+xe; ybe= yb+ye; subplot(224); plot2d(xbe,ybe,-2); // second row of ndt subplot(224); xred= [ndt(2:2,1:2*nr)] ;//reduced dimension taking first row of ndt back= V(1:2,2:2)*xred ; xb=back(1:1,1:2*nr) ; yb=back(2:2,1:2*nr) ; plot2d(xb,yb,-2);
xbe= xb+xe; ybe= yb+ye; subplot(224); plot2d(xbe,ybe,-2);
//xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 3. Hankel PCA and ERA on two sine waves ( to determine the frequencies from the sine outputs.) Now let us apply it to Hankel as follows: 1.We obtain data as data=8*sin(1.6*%pi*t) +1*sin(16*%pi*t) ; 2. Then we obtain centered (subtract mean in each column) Hankel and Covariant matrix from h2c=H0-he;// centered zero mean hankel matrix c=h2c*h2c' ;// cov matrix 3. The Change of basis into eigen matrix is obtained from [ei,dg]= spec(c);//dg=eig values and ei=eig matrix 4. The new hankels are : h1c=H1-he;// centered zero mean hankel matrix xe1=ei*h1c ; (h2c similarly ) H0=xei; H1= xe1; 5 The ERA is then applied
we get diag (S) // 4 non zero eigen values. ans S = eigen values of ERA . 178.89542 111.56389 19.199544 15.032411 5.614D-14 4.750D-14 3.973D-14 3.608D-14 6.If we set n=4 and ERA then //eig_Ac //
=
5.690D-15 + 5.0265482i
//
5.690D-15 - 5.0265482i
//
1.693D-14 + 50.265482i
//
1.693D-14 - 50.265482i
// compare input 2*%pi*.8 =
5.0265482
// 2*%pi*8= 50.265482 so result is ok.. //xxxxxxxxxxxxxxxxxxxxxxxxxx
1. Recall Hankel and ERA. //Expt 1 Fs=40
t=0:1/Fs:20; // time axis from 0 to 20 sec lam =-.05 //damping constant //lam=0 //data=8*sin(1.6*%pi*t) .*(exp(lam*t));// here ampl=8 f0=.8hz data=8*sin(1.6*%pi*t) +1*sin(16*%pi*t) ; figure(0); plot(t,data); //title ("damped sinusoid"); length(t) //101. x=data ; //let us do ERA on data abc=x; // size(x) = 1. 801. //abc= sqrt(psc);// sqrt because signal squared gives energy not used here abc_new=abc(4:80); size(abc_new)//1.
77.
Fsn=Fs N=size(abc_new,2); ns=1; N_act=round(N/2-1); // N_act=20; no_y=2; // ax=ym(:,1:11);N=size(ax,2); c=0; H0=[]; // N_act=20; for i=1:N_act ic=i*ns+1; y_int=abc_new(ic:N_act+ic-1)'; H0=[H0 y_int] ; end size(H0)// 38. 38. H1=[]; for i=1:N_act ic=i*ns+1+ns; y_int=abc_new(ic:N_act+ic-1)'; H1=[H1 y_int] ; end [U S V]=svd(H0); n=2;// changed from 1 Sn=S(1:n,1:n); Un=U(:,1:n); Vn=V(:,1:n); dis_A=inv(sqrt(Sn))*Un'*H1*Vn*inv(sqrt(Sn)); eigen_dis_A=spec(dis_A); // discrete system eigen values eig_Ac=log(eigen_dis_A)/(1/Fsn); // continuous system eigen values // f=abs(eig_Ac)/(2*%pi);
eig_Ac diag(S)// 180.21413 //
6.395D-14
// eig_Ac
=
122.50146 5.868D-14
19.288664 18.629212 7.243D-14 5.095D-14
6.568D-14
// //
0.0052602 + 5.0287591i 0.0052602 - 5.0287591i
//for .8hz omega= 2pi*.8 =
5.0265482
so ok
// if we set n=4 we get n=4;// changed from 1 Sn=S(1:n,1:n); Un=U(:,1:n); Vn=V(:,1:n); dis_A=inv(sqrt(Sn))*Un'*H1*Vn*inv(sqrt(Sn)); eigen_dis_A=spec(dis_A); // discrete system eigen values
eig_Ac=log(eigen_dis_A)/(1/Fsn); //eig_Ac //
=
5.690D-15 + 5.0265482i
//
5.690D-15 - 5.0265482i
//
1.693D-14 + 50.265482i
//
1.693D-14 - 50.265482i
// compare input 2*%pi*.8 =
5.0265482
// 2*%pi*8= 50.265482 so result is ok.. //xxxxxxxxxxxxxxxxxxxxxxxxxxxx
// Expt 2 apply PCA on Hankel Fs=40 t=0:1/Fs:20; // time axis from 0 to 20 sec lam =-.05 //damping constant //lam=0 //data=8*sin(1.6*%pi*t) .*(exp(lam*t));// here ampl=8 f0=.8hz
data=8*sin(1.6*%pi*t) +1*sin(16*%pi*t) ; figure(0); plot(t,data); //title ("damped sinusoid"); length(t) //101. x=data ; //let us do ERA on data abc=x; // size(x) = 1. 801. //abc= sqrt(psc);// sqrt because signal squared gives energy not used here abc_new=abc(4:80); size(abc_new)//1.
77.
Fsn=Fs N=size(abc_new,2); ns=1; N_act=round(N/2-1); // N_act=20; no_y=2; // ax=ym(:,1:11);N=size(ax,2); c=0; H0=[]; // N_act=20; for i=1:N_act ic=i*ns+1; y_int=abc_new(ic:N_act+ic-1)'; H0=[H0 y_int] ; end size(H0)// 38. 38. H1=[]; for i=1:N_act ic=i*ns+1+ns; y_int=abc_new(ic:N_act+ic-1)'; H1=[H1 y_int] ;
end //xxxxxxxxxxxxxxxxxxxxxx contd // of Prinipal Comp Analysis viz eig basis nn= size(H0,2) // set size of he hmn=sum(H0,1); //add cols hmn=hmn/nn; // find col mean he=[] for i=1:nn he=[ he hmn'] ; end he =he' ;// matrix of means hmn=sum(H0,1); //add cols hmn=hmn/nn; // find col mean he=[] for i=1:nn he=[ he hmn'] ; end he =he' ;// matrix of means size(he)
h2c=H0-he;// centered zero mean hankel matrix c=h2c*h2c' ;// cov matrix [ei,dg]= spec(c);//dg=eig values and ei=eig matrix [dgs ]= diag(dg); // vectorize dg size(ei) xei = ei*h2c ;// change variables into ortho basis // of Prinipal Comp Analysis viz eig basis hmn=sum(H1,1); //add cols hmn=hmn/nn; // find col mean he=[] for i=1:nn he=[ he hmn'] ; end he =he' ;// matrix of means hmn=sum(H1,1); //add cols hmn=hmn/nn; // find col mean he=[] for i=1:nn he=[ he hmn'] ; end he =he' ;// matrix of means size(he) h1c=H1-he;// centered zero mean hankel matrix xe1=ei*h1c ; H0=xei; H1= xe1; [U S V]=svd(H0); n=2;// changed from 1 with n=4 we get correct result Sn=S(1:n,1:n); Un=U(:,1:n); Vn=V(:,1:n); dis_A=inv(sqrt(Sn))*Un'*H1*Vn*inv(sqrt(Sn)); eigen_dis_A=spec(dis_A); // discrete system eigen values
eig_Ac=log(eigen_dis_A)/(1/Fsn) // with n=2 eig_Ac
=
- 0.0318036 + 5.0382114i - 0.0446606 - 5.0394002i
// with n=4 eig_Ac
=
4.094D-14 + 5.0265482i 2.005D-14 - 5.0265482i - 3.497D-14 + 50.265482i - 6.106D-15 - 50.265482i
diag (S) ans
// 4 non zero eigen values.
=
178.89542 111.56389 19.199544 15.032411 5.614D-14 4.750D-14 3.973D-14 3.608D-14 3.292D-14 //compare with earlier no PCA applied.. >diag(S) ans
=
180.21413 122.50146 19.288664 18.629212
7.243D-14 6.568D-14 6.395D-14
