Sensitivity of roots to errors in the coefficient of ... - CiteSeerX

1050

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 38, NO. 6, DECEMBER 1989

Sensitivity of Roots to Errors in the Coefficient of Polynomials Obtained by Frequency-Domain Estimation Methods PATRICK GUILLAUME, MEMBER, IEEE,J. SCHOUKENS.

Abstract-It is a well-known fact that the roots of a polynomial of high order are extremely sensitive to perturbations in its coefficients. Nevertheless, experience has demonstrated that frequency-domain estimation techniques succeed in the determination of accurate poles and zeros, even in the case of high-order transfer function models. In this paper we prove that this is due to the correlations among the estimated coefficients. When the result of a measurement is a set of correlated values, we conclude that it is not justifiable to use the standard deviation to determine the number of significant digits. Additional digits have to be considered in order to maintain the information enclosed in the correlations. Index Terms-Parameter estimation, zeros of polynomials, correlation, round off errors.

I. INTRODUCTION EN the coefficients of a polynomial are obtained from measurements, one cannot expect to find the true coefficient values. There will always be a more or less important discrepancy depending on the accuracy of the measurements. Frequency-domain parameter estimation is no exception, and the estimated coefficients will be contaminated by measurement errors. The impact of these errors on the roots of an estimated polynomial will be examined in this paper. We shall see that the errors in the estimated coefficients can be strongly correlated, and that these correlations are the reason why the roots can still be determined accurately in the case of high-order transfer function. The sensitivity of the roots to uncorrelated errors in the coefficients of a polynomial is not a new problem. It has been treated some time ago by, among others, Ralstone [l] and Wilkinson [2]. Their theory is summarized in the next section. In the remainder of this text we shall refer to it as the classical approach. In Section I11 we apply this theory to a polynomial of practical interest. By means of simulations we show in Section IV that the estimated polynomials have roots which are much closer to the exact solution than is predictable by applying the classical the-

ory. Next, the existence of strong correlations among the estimated coefficients is demonstrated. In Section V the classical formula is expanded to take into account these correlations. We prove that the sensitivity of the roots to correlated errors in the coefficients is always less important than predicted by the classical approach. Finally, notable remarks are made about the effect of round off errors on correlated data. When the result of a measurement is a set of correlated data, it will be shown that the number of significant digits cannot be determined by means of the standard deviation only. More digits must be included to maintain the information contained in the correlations. OF ROOTSTO UNCORRELATED 11. SENSITIVITY COEFFICIENT ERRORSOF A POLYNOMIAL (THE CLASSICAL APPROACH) A polynomial with n zeros can be written as

w

Manuscript received January 12, 1989; revised May 5 , 1989. The work of J . Schoukens and R. Pintelon was supported by the National Fund for Scientific Research, Belgium. The authors are with the Electrical Measurement Department (ELEC), Vrije Universiteit Brussel, 1050 Brussels, Belgium. IEEE Log Number 8931 180.

RIK PINTELON

AND

n

f(s) =

c

(2.1)

UkSk

k=O

or n

f(s) = K

II i= 1

(S

- s;),

K =

U,

(2.2)

and

f (Si) = 0

(2.3)

for all roots si,i = 1 , 2, * . . n . When a perturbation Auk is applied to one of the coefficients uk, (2.1) becomes 9

f ( s ) =f(s)

+ Auksk.

(2.4)

The roots of (2.4) differ from si by A s i , which may be real or complex:

f(sj

+ AS;) = f ( s i + A s ; ) + A u ~ ( s+; AS^)^ = 0 . (2.5)

The above equation can be simplified by applying Taylor’s formula and assuming that Asi and Auk are sufficiently small so that all products of the perturbations can be neglected. Equation (2.5), therefore, becomes

+

f ’ ( s j ) A s j Auksf = 0.

0018-9456/89/1200-1050$01.OO 0 1989 IEEE

Authorized licensed use limited to: Rik Pintelon. Downloaded on November 25, 2008 at 03:28 from IEEE Xplore. Restrictions apply.

(2.6)

GUILLAUME et al.: ROOTS TO ERRORS

IN THE COEFFICIENT OF POLYNOMIALS

Equation ( 2 . 6 ) can be expressed in a sensitivity formula as

1051

TABLE 1

M A X I M AALLOWED L RELATIVE ERRORS I N THE COEFFICIENTS OF g, (s) TO 1-PERCENT ERROR IN THE ROOTS

(2.7) V

where S:k, the sensitivity of the relative error on the root si to the relative perturbation of the coefficient ak, equals

1 - 1

I

12 14

I'

n

Kll

10

(Si

-

5.7E-06 6.OE-07 7.1E-08

TABLE I1 TRUEA N D ESTIMATED COEFFICIENTS OF g,, (s)

Sj)J

jzi

The maximal value of the sensitivity will be denoted by = Max (SP ) . One obvious limitation of this approximation occurs whenf ' ( s i ) is zero (e.g., multiple roots) or very small, in which case the assumption that higher order terms of the Taylor's expansion could be neglected was unfounded.

k 0 2 4 6

8 10

I2 14 16 18

111. EXAMPLE OF

1744 16577 141676

PRACTICAL ILL-CONDITIONED POLYNOMIAL Consider for instance the following ill-conditioned polynomial of order 2v, with zeros equally spaced along the imaginary axis: A

20 22 24 26 28

True Coeficients 7.60005445655199lE+2 1 1.19776541997034lE+22 5.325923338791614E+21 1.054368810603158E+21 lI13057O8629285oOE+20 7.307216948928239E+18 3.036268070760506E+17 8.4396547589702258+ 15 1.605575083448550E+14 2.109665355525000E+12 1.9O58554515(nloOoE+10 1.158426750000000E+08 4,5126-E+05 1.01SMXXKWXXXKMOE+03 I-,E+CCI

Estimated Coefficients

7.589975941 349581E+21 1.1962232832974918+22 5.319534157456828E+21 1.053150919668680E+21 1.129287064891008E+20 7.29895473548 1372E+18 3.032823588408874E+17 8.430009095453 187E+15 1.603722749370390E+14 2.107206550704012E+12 1.903611043847356E+lO I. 157048438740264E+08 4.507266095098 199E+05 1.0137678 17308401E+03 9.987739901223392E-01

fictitious system. This system was excited by a multisine constructed by summing 300 sine waves with the same amplitude with frequencies uniformly spaced between This could be the characteristic polynomial of an un- 0.145 and 2.28 Hz. The calculated Fourier coefficients of damped mechanical system with v degrees of freedom. the input and output signal were contaminated by zero Table I shows that to have relative errors smaller than 1 mean Gaussian noise with a standard deviation equal to 1 percent in the roots, the coefficients of g I 4 ( s )must be percent of their amplitude. These polluted Fourier coefaccurate to at least seven decimal digits ( g I 4 ( s )stands ficients were then used to estimate the polynomial gl4 (s) for the polynomial form (2.1) of (3.1) with v equal to 14). with a maximum likelihood estimator [ 5 ] . All odd order This is practically impossible when dealing with experi- coefficients of g14(s) were assumed to be zero. The remental data. From Table I, it looks as if only bands with sults of simulation A are summarized in Tables I1 and 111. From Table 111, it seems that we can indeed get quite less than 10 modes seem to be feasible. However, experience has demonstrated that frequency- accurate roots from the estimated 28th-order polynomial domain estimation techniques succeed in the determina- without knowing seven correct digits of the coefficients. tion of accurate poles and zeros, even in the case of high- If you take a look at Table I1 and compare the estimated order transfer functions. To corroborate this, the follow- coefficients with the true values you will see that the estimated coefficients have fewer than five correct signifiing simulations were carried out. cant digits. Thus one could expect that the roots of the IV. SIMULATIONS polynomial obtained by truncating the true coefficients Because we are examining the influence of coefficient after the fifth digit should be better than the roots obtained errors on the roots of a polynomial, we have limited the from the estimated polynomial. Table 111 shows that this classical transfer function model to a polynomial by set- is not the case. Indeed, as mentioned in Section 111, we ting its denominator equal to one in all subsequent simu- need to maintain at least seven digits to obtain an acculations. This does not affect the generality of our conclu- racy of 1 percent in the root estimates. The estimated sions when estimating transfer functions. Indeed, the zeros polynomial clearly does not satisfy this rule. Its root are of a transfer function only depend on the numerator coef- much better than could be expected from Table I. So, alficients and the poles only on the denominator coeffi- though the coefficient perturbations are much larger than cients. The generalization of the problem to transfer func- allowed by the classical approach, the roots generated are tion models is obvious. quite good. Fig. 1 illustrates the impact of estimation errors and A. Estimation of the Polynomial 814 (s) round off errors on the amplitude of g l l(s). The estiIn simulation A the ill-conditioned polynomial gI4 ( s ) mated curve is a much better fit to the true values than of order 28 has been used as the transfer function of a that provided by the truncated polynomial. V

&(S)

=

II (=I

(s2+

t2).

(3.1)

Authorized licensed use limited to: Rik Pintelon. Downloaded on November 25, 2008 at 03:28 from IEEE Xplore. Restrictions apply.

1052

IEEE TRANSACTIONS ON INSTRUMENTATION A N D MEASUREMENT, VOL. 38. NO. 6 , DECEMBER 1989

0

3

6

9

15

12

Pulsation (rad/s)

Fig. 1 . Amplitude characteristic of the function g , 4 ( s ) computed with: the exact coefficient values, the estimated coefficient values, and rn the exact values truncated after the fifth significant digit.

+

TABLE 111 COMPARISON OF T H E TRUEROOTS, T H E ROOTSOF T H E ESTIMATED POLYNOMIAL &'14(S). AND THE ROOTSOF THE TRUEPOLYNOMIAL TRUNCATED AFTERFIVEDIGITS True Roo&

O f j1 OfjZ Ofj3 Ofj4 Oij5 Ofj6 Ofj7 Oij8 Ofj9 O f j IO O f j 11 O f j 12

O f j 13 O f j 14

Rootsof the EstimatedPolynomial

0 f j 1.0000062 O i j 1.9996691 0 f j 3.0001256 0 f j 3.9998404 0 f j 5.0000673 0 f j 6.0002927 0 f j 7.0001362 0 f j 8.0000014 0 f j 8.9999760 O f j 10.000113 0 f j 10.999963 0 f j 12.oooO22 0 f j 13.000155 0 f j 14.oooO83

Roots of the Polynomial with Truncated Coeff. 0 f j 1.oooO783 0 f j 1.9990500 0 f j 3.0010401 0 f j 4.0581740 0 f j 4.7122291 0.9144404 f j 6.1032296 -0.9144404 f j 6.1032296 1.6876474f j 8.1614494 -1.6876474fj 8.1614494 1.8647555 f j 10.651625 -1.8647555 f j 10.651625 1.2735444 f j 13.000566 -1.2735444 f j 13.000566 0 f j 14.395472

1.006

1.004

g

1.002

g

1.000

U

0.998 0.996 0.996 0.998 1.ooO 1.002 1.004 1.006 Coefficient

a,

(a) 1.06

1.04

E I

1.02

E

3 8

1.00 0.98 0.96 0.94 6 Coefficient

(b)

B. Estimation of the Polynomial gl (s) Simulations B1, B2, and B3 use the second-order polynomial g l ( s ) as the transfer function of the DUT. The excitation consists of a multisine composed of 20 sine waves with frequencies equally spaced between 0 and 0.3 Hz. The exact Fourier coefficients of the input and output signal are contaminated by zero mean Gaussian noise with standard deviations equal to 1 percent ( B l ) , 10 percent ( B 2 ) , and 100 percent ( B 3 ) of their amplitude. These Fourier coefficients are then used to estimate the coefficients a. and a 2 of g, (s) . Coefficient a , is presumed to be zero. These simulations were repeated 100 times. Fig. 2(a)-(c) illustrate the strong correlations among the estimated coefficients. All the points ( ao, a 2 )are confined on a straight line going through the origin and the exact solution ( 1, 1 ). All points lying on this path give correct roots. Thus it is not the distance from ( ao, a 2 )to ( 1, 1 ) (number of correct digits of the coefficients) which determines the accuracy of the roots but rather the distance

2.0 E

1.5

8

1.0

8

0.5

I

0.0 0.0

0.5

1.0 Coefficient

1.5

2.0

a,

(C) Fig. 2. Estimated coefficients of g , (s) for a noise level of (a) 1 percent, (b) 10 percent, and (c) 100 percent.

from ( a o , a 2 ) to this line. The discrepancy between the two distances can be very important when dealing with correlations. This partially explains why frequency-domain estimation techniques do not suffer as much from ill-conditioning as the classical approach makes us believe.

Authorized licensed use limited to: Rik Pintelon. Downloaded on November 25, 2008 at 03:28 from IEEE Xplore. Restrictions apply.

GUILLAUME er al.: ROOTS TO ERRORS IN THE COEFFICIENT

OF POLYNOMIALS

1053

V . THE EFFECTOF CORRELATIONS ON THE SENSITIVITY comes OF THE ROOTS In this section we derive a relation between the covari- with ance matrix of the unknown polynomial coefficients and the covariance matrix of all the independent real and imaginary parts of its roots. For this purpose we introduce a vector T which has the and unknown coefficients of (2.1) (e.g., ak with k = w l , wz, , w N )as entries: .

I

A 9 , = F,Ar

+r

=

(5.6)

Ft

=

(5.7)

.

TT

= [U,,,,

(5.1)

U,,]

where N is the number of unknown coefficients. A vector 4, contains the real and imaginary parts of all the roots si in (2.2) with imaginary parts greater than or equal to zero:

4;

= [Re

(sl)Im (s,) Re

(',+I)

*

Re

('K+CY)

Im

n-k

ak =

* *

K C a

(',+,+I)

* *

*

lm(sK+CY+fl)]

(5.2) and number of complex roots with imaginary part > 0 and real part f 0, CY number of purely real roots, /3 number of purely imaginary roots with imaginary part > 0. K

We also define a sensitivity matrix S, = [ S,,] as if u is odd and 1

-

where e l = [0 * * 1 * * * 01 with the 1 entry at the same place as a, in r ( n = wg,see (5.1)). Equations (2.1) and (2.2) both contain IZ + 1 real parameters. Suppose that m of the coefficients ak are known a priori. Then, there exist m relations among the parameters of the factored form (2.2). These m relations are

Re (s,) Im (s,)

*

IU

(5.5)

< ~K:S,,,

II

Sa,

(i=l

)

(5.8)

where k equals the index of the known coefficients, and the sum is over the ( ! ! k ) possible combinations of the n roots, taken ( n - k ) at a time. Equation (5.8) may be linearized if 11 A 9 , I( is sufficiently small. In this way, we obtain a set of m linear 1 entries of A+, as unknown variequations with the n ables. Thus only N = ( n 1 - m ) entries of A @ , are linearly independent. Let us construct a vector A 9 by eliminating m dependent parameters of A+, , and a matrix F by removing the corresponding rows of F,. Then, A 9 and AT both have N entries and F is a square matrix

+

+

A 9 = FAT Lemma 1: F is a regular matrix.

(5.9)

Proof: See Appendix

i f u isevenand 1


. I

\

(6.3)

Using (5.10), one can find an approximation of the covariance matrix A* of the roots. Table IV contains an estimation of the Cramer-Rao lower bound standard devia). The Cramer-Rao matrix of the tion of the roots ( u~iprox. roots has been found by replacing A+ in (5.10) by the Cramer-Rao lower bound matrix of the coefficients. Simulation A was executed 20 times ( NExp.= 20) . The empirical values of the standard deviations (uEmp.)are then given by

{ Irn ( s i ]

After linearization of (6.2) we find that AK=O A Re ( s 2 ) = -A Re ( s , ) .

(6.4)

So, the variances of the dependent variables are var { K } =

o

var {Re ( s 2 ) } = var {Re ( S I ) } = var { a , } .

(6.5) To find the covariance matrix of the other variables, it is sometimes easier to apply (5.10) again for another set of independent variables 9, e.g.:

B. Comparison of the Empirical and Approximated Standard Deviations of the Roots of g14(s) Obtained from Simulation A . For simulation A , the vectors AT and A+ are

[:' 1 1. A Im

AT

=

A9

=

(SI)

1m(s2)

A Im

Aa28

with XEmp. { Im ( s i ) } the arithmetic mean of Im ( s i ) . Due to the large amount of data (300 spectral lines) processed by our maximum likelihood estimator we can be sure that the estimates of the coefficients have reached their asymptotic values. Statistics then tells us that the /u~~pm has x .a probability of 90 percent to occur ratio uEmp. in the interval (0.736, 1.25). Conclusion: From Table IV, it follows that the Cramer-Rao under bound matrix calculated by (5.10) coincides completely with the empirical values. Thus the variances of the root estimates have also reached their Cramer-Rao under bounds. This demonstrates that these roots are the most accurate ones, obtainable from our data and a priori knowledge. So, the computation of the roots did not introduce a supplemental error. The correlations among the coefficients make the ill-condition polynomial much less sensitive to coefficient errors.

(SI41

AK

(6.7)

VII. INFLUENCE OF ROUND OFF ERRORSON CORRELATED DATA Suppose you have extracted the transfer function of a DUT from measured input and output signals by means of a frequency-domain parameter estimation program. The estimated coefficients of the transfer function will be con-

Authorized licensed use limited to: Rik Pintelon. Downloaded on November 25, 2008 at 03:28 from IEEE Xplore. Restrictions apply.

GUILLAUME

er

d.:ROOTS TO ERRORS I N THE COEFFICIENT

OF POLYNOMIALS

taminated by measurement inaccuracies and the noise generated by the DUT itself. To decide how many digits are significant one normally relies on the standard deviation of the estimated coefficients. However, this method can lead to serious waste of information about the correlations. Many more digits usually have to be considered even if these are known to be incorrect. Indeed, rounding off or truncating the coefficients will introduce uncorrelated noise. This additional noise will sum to the diagonal elements of the covariance matrix of the coefficients. Thus the nondiagonal elements will become less meaningful and some information contained in the correlations will be lost. Even small values added to the diagonal elements of A, can cause a significant increase of its determinant and thus also of the determinant of A*. This will result in important perturbations of the roots. Because they are uncorrelated, round off and truncation errors obey the rule of Section I1 and thus can cause an important deterioration in the accuracy of the roots. The sensitivity factor SMAx can be used to determine the minimal number of digits required to maintain enough correlations among the coefficients to obtain roots with an accuracy of for example 1 percent. Let us again consider simulation A . From Fig. 1 it is obvious that all the roots of the estimated polynomial are in the immediate vicinity of the correct roots. Indeed, the estimation algorithm forces the estimated transfer function to fit the measured one. If the position of the poles and zeros is apparent in the measured transfer function, the roots of the estimated polynomials are well defined even if the estimated coefficients are not very accurate. Consider the vector A7, containing the relative error of the unknown coefficients

and the vector A+, with the relative error of the independent roots as entries. Then, (5.10) becomes

A+,.

=

F,Ar,.

(7.2)

The Euclidean norm or 2-norm of A@, is given by

To allow only small variations of the roots, (7.3) must be limited:

II

II 2