Sensitivity of Roots to Errors in the Coefficient of Polynomials Obtained by Frequency -Domain Estimation Methods

Transcription

1 1050 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 38, NO. 6, DECEMBER 1989 Sensitivity of Roots to Errors in the Coefficient of Polynomials Obtained by Frequency -Domain Estimation Methods PATRICK GUILLAUME, MEMBER, IEEE, J. SCHOUKENS. AND RIK PINTELON Abstract-It is a well-known fact that the roots of a polynomial of high order are extremely sensitive to perturbations in its coefficients. Nevertheless, experience has demonstrated that frequency-domain estimation techniques succeed in the determination of accurate poles and zeros, even in the case of high-order transfer function models. In this paper we prove that this is due to the correlations among the estimated coefficients. When the result of a measurement is a set of correlated values, we conclude that it is not justifiable to use the standard deviation to determine the number of significant digits. Additional digits have to be considered in order to maintain the information enclosed in the correlations. Index Terms-Parameter estimation, zeros of polynomials, correlation, round off errors. w I. INTRODUCTION EN the coefficients of a polynomial are obtained from measurements, one cannot expect to find the true coefficient values. There will always be a more or less important discrepancy depending on the accuracy of the measurements. Frequency-domain parameter estimation is no exception, and the estimated coefficients will be contaminated by measurement errors. The impact of these errors on the roots of an estimated polynomial will be examined in this paper. We shall see that the errors in the estimated coefficients can be strongly correlated, and that these correlations are the reason why the roots can still be determined accurately in the case of high-order transfer function. The sensitivity of the roots to uncorrelated errors in the coefficients of a polynomial is not a new problem. It has been treated some time ago by, among others, Ralstone [l] and Wilkinson [2]. Their theory is summarized in the next section. In the remainder of this text we shall refer to it as the classical approach. In Section I11 we apply this theory to a polynomial of practical interest. By means of simulations we show in Section IV that the estimated polynomials have roots which are much closer to the exact solution than is predictable by applying the classical the- Manuscript received January 12, 1989; revised May 5, The work of J. Schoukens and R. Pintelon was supported by the National Fund for Scientific Research, Belgium. The authors are with the Electrical Measurement Department (ELEC), Vrije Universiteit Brussel, 1050 Brussels, Belgium. IEEE Log Number ory. Next, the existence of strong correlations among the estimated coefficients is demonstrated. In Section V the classical formula is expanded to take into account these correlations. We prove that the sensitivity of the roots to correlated errors in the coefficients is always less important than predicted by the classical approach. Finally, notable remarks are made about the effect of round off errors on correlated data. When the result of a measurement is a set of correlated data, it will be shown that the number of significant digits cannot be determined by means of the standard deviation only. More digits must be included to maintain the information contained in the correlations. or and 11. SENSITIVITY OF ROOTS TO UNCORRELATED COEFFICIENT ERRORS OF A POLYNOMIAL (THE CLASSICAL APPROACH) A polynomial with n zeros can be written as n f(s) = c UkSk (2.1) k=o n f(s) = K II (S - s;), K = U, (2.2) i= 1 f (Si) = 0 (2.3) for all roots si, i = 1, 2, *.. 9 n. When a perturbation Auk is applied to one of the coefficients uk, (2.1) becomes f(s) =f(s) + Auksk. (2.4) The roots of (2.4) differ from si by Asi, which may be real or complex: f(sj + AS;) =f(si + As;) + Au~(s; + AS^)^ = 0. (2.5) The above equation can be simplified by applying Taylor s formula and assuming that Asi and Auk are sufficiently small so that all products of the perturbations can be neglected. Equation (2.5), therefore, becomes f (sj)asj + Auksf = 0. (2.6) /89/ $01.OO IEEE

2 GUILLAUME et al.: ROOTS TO ERRORS IN THE COEFFICIENT OF POLYNOMIALS 1051 Equation (2.6) can be expressed in a sensitivity formula as (2.7) where S:k, the sensitivity of the relative error on the root si to the relative perturbation of the coefficient ak, equals TABLE 1 MAXIMAL ALLOWED RELATIVERRORS IN THE COEFFICIENTS OF g, (s) TO 1 -PERCENT ERROR IN THE ROOTS V E OE E n I' Kll (Si - S j)j I jzi The maximal value of the sensitivity will be denoted by = Max (SP ). One obvious limitation of this approximation occurs whenf ' (si) is zero (e.g., multiple roots) or very small, in which case the assumption that higher order terms of the Taylor's expansion could be neglected was unfounded EXAMPLE OF A PRACTICAL ILL-CONDITIONED POLYNOMIAL Consider for instance the following ill-conditioned polynomial of order 2v, with zeros equally spaced along the imaginary axis: V &(S) = II (s2 + t2). (3.1) ( = I This could be the characteristic polynomial of an undamped mechanical system with v degrees of freedom. Table I shows that to have relative errors smaller than 1 percent in the roots, the coefficients of gi4(s) must be accurate to at least seven decimal digits ( gi4(s) stands for the polynomial form (2.1) of (3.1) with v equal to 14). This is practically impossible when dealing with experimental data. From Table I, it looks as if only bands with less than 10 modes seem to be feasible. However, experience has demonstrated that frequencydomain estimation techniques succeed in the determination of accurate poles and zeros, even in the case of highorder transfer functions. To corroborate this, the following simulations were carried out. IV. SIMULATIONS Because we are examining the influence of coefficient errors on the roots of a polynomial, we have limited the classical transfer function model to a polynomial by setting its denominator equal to one in all subsequent simulations. This does not affect the generality of our conclusions when estimating transfer functions. Indeed, the zeros of a transfer function only depend on the numerator coefficients and the poles only on the denominator coefficients. The generalization of the problem to transfer function models is obvious. A. Estimation of the Polynomial 814 (s) In simulation A the ill-conditioned polynomial gi4 (s ) of order 28 has been used as the transfer function of a TABLE I1 TRUE AND ESTIMATED COEFFICIENTS OF g,, (s) k True Coeficients Estimated Coefficients I lE lE E E+21 li13057o ooe E E E E O (nloOoE E+08 4,5126-E SMXXKWXXXKMOE+03 I,-E+CCI E E E E E E E E E E+lO I E E E E-01 fictitious system. This system was excited by a multisine constructed by summing 300 sine waves with the same amplitude with frequencies uniformly spaced between and 2.28 Hz. The calculated Fourier coefficients of the input and output signal were contaminated by zero mean Gaussian noise with a standard deviation equal to 1 percent of their amplitude. These polluted Fourier coefficients were then used to estimate the polynomial gl4 (s) with a maximum likelihood estimator [5]. All odd order coefficients of g14(s) were assumed to be zero. The results of simulation A are summarized in Tables I1 and 111. From Table 111, it seems that we can indeed get quite accurate roots from the estimated 28th-order polynomial without knowing seven correct digits of the coefficients. If you take a look at Table I1 and compare the estimated coefficients with the true values you will see that the estimated coefficients have fewer than five correct significant digits. Thus one could expect that the roots of the polynomial obtained by truncating the true coefficients after the fifth digit should be better than the roots obtained from the estimated polynomial. Table 111 shows that this is not the case. Indeed, as mentioned in Section 111, we need to maintain at least seven digits to obtain an accuracy of 1 percent in the root estimates. The estimated polynomial clearly does not satisfy this rule. Its root are much better than could be expected from Table I. So, although the coefficient perturbations are much larger than allowed by the classical approach, the roots generated are quite good. Fig. 1 illustrates the impact of estimation errors and round off errors on the amplitude of gll (s). The estimated curve is a much better fit to the true values than that provided by the truncated polynomial.

3 1052 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 38. NO. 6, DECEMBER Pulsation (rad/s) Fig. 1. Amplitude characteristic of the function g,4(s) computed with: - the exact coefficient values, + the estimated coefficient values, and rn the exact values truncated after the fifth significant digit. TABLE 111 COMPARISON OF THE TRUE ROOTS, THE ROOTS OF THE ESTIMATED POLYNOMIAL &'14(S). AND THE ROOTS OF THE TRUE POLYNOMIAL TRUNCATED AFTER FIVE DIGITS True Roo& Rootsof the Roots of the Polynomial Estimated Polynomial with Truncated Coeff. Ofj 1 OfjZ Ofj3 Ofj4 Oij5 Ofj6 Ofj7 Oij8 Ofj9 O f j IO Ofj 11 Ofj 12 Ofj 13 O f j 14 0 f j Oij f j f j f j f j f j f j f j Ofj f j f j 12.oooO22 0 f j f j 14.oooO83 0 f j 1.oooO783 0 f j f j f j f j f j f j f j fj f j f j f j f j f j B. Estimation of the Polynomial gl (s) Simulations B1, B2, and B3 use the second-order polynomial gl (s) as the transfer function of the DUT. The excitation consists of a multisine composed of 20 sine waves with frequencies equally spaced between 0 and 0.3 Hz. The exact Fourier coefficients of the input and output signal are contaminated by zero mean Gaussian noise with standard deviations equal to 1 percent (Bl), 10 percent (B2), and 100 percent (B3) of their amplitude. These Fourier coefficients are then used to estimate the coefficients a. and a2 of g, (s). Coefficient a, is presumed to be zero. These simulations were repeated 100 times. Fig. 2(a)-(c) illustrate the strong correlations among the estimated coefficients. All the points ( ao, a2) are confined on a straight line going through the origin and the exact solution ( 1, 1 ). All points lying on this path give correct roots. Thus it is not the distance from ( ao, a2) to ( 1, 1 ) (number of correct digits of the coefficients) which determines the accuracy of the roots but rather the distance g g U ooO Coefficient a, E I 1.02 E E 1.5 I (a) Coefficient (b) Coefficient a, (C) Fig. 2. Estimated coefficients of g, (s) for a noise level of (a) 1 percent, (b) 10 percent, and (c) 100 percent. from (ao, a2) to this line. The discrepancy between the two distances can be very important when dealing with correlations. This partially explains why frequency-domain estimation techniques do not suffer as much from ill-conditioning as the classical approach makes us believe. 6

4 GUILLAUME er al.: ROOTS TO ERRORS IN THE COEFFICIENT OF POLYNOMIALS 1053 V. THE EFFECT OF CORRELATIONS ON THE SENSITIVITY comes OF THE ROOTS In this section we derive a relation between the covari- with ance matrix of the unknown polynomial coefficients and the covariance matrix of all the independent real and imaginary parts of its roots. For this purpose we introduce a vector T which has the and unknown coefficients of (2.1) (e.g., ak with k = wl, wz,. I., wn) as entries: A9, = F,Ar (5.5) +r = Ft = (5.6) (5.7) TT = [U,,,, U,,] (5.1) where N is the number of unknown coefficients. A vector 4, contains the real and imaginary parts of all the roots si in (2.2) with imaginary parts greater than or equal to zero: and 4; = [Re (sl) Im (s,) * Re (s,) Im (s,) Re (',+I) * * * Re ('K+CY) Im (',+,+I) * * * lm(sk+cy+fl)] (5.2) K number of complex roots with imaginary part > 0 and real part f 0, CY number of purely real roots, /3 number of purely imaginary roots with imaginary part > 0. We also define a sensitivity matrix S, = [ S,,] as if u is odd and 1 I U < ~K:S,,, ifu isevenand 1 < U I ~K:S,, where k equals the index of the coefficient on the vth entry of r(k = w,). So, the relation A& StAr (5.4) is a trivial extension of (2.6). As in Section 11, we assume that there are no multiple roots ( f ' (si ) # 0). From (2.2), we see that A K = Aa,. Thus the variations of K can easilv be taken into account. Eauation (5.4) be- where el = [0 - * * 1 * * * 01 with the 1 entry at the same place as a, in r(n = wg, see (5.1)). Equations (2.1) and (2.2) both contain IZ + 1 real parameters. Suppose that m of the coefficients ak are known a priori. Then, there exist m relations among the parameters of the factored form (2.2). These m relations are n-k ak = K C II Sa, a ( i = l ) (5.8) where k equals the index of the known coefficients, and the sum is over the (!!k) possible combinations of the n roots, taken (n - k ) at a time. Equation (5.8) may be linearized if 11 A9, I( is sufficiently small. In this way, we obtain a set of m linear equations with the n + 1 entries of A+, as unknown variables. Thus only N = (n m) entries of A@, are linearly independent. Let us construct a vector A9 by eliminating m dependent parameters of A+,, and a matrix F by removing the corresponding rows of F,. Then, A9 and AT both have N entries and F is a square matrix A9 = FAT (5.9) Lemma 1: F is a regular matrix. Proof: See Appendix The covariance matrix of the elements present in the vector 9 can be approximated by A* = E{A9A@'} = E ( F A ~ A ~ ~ F ~ } = FE{ AT AT^} FT = FA,F~. (5.10) Thus: la I = lf121atl. (5.11) Lemma 2: The determinant of a positive definite matrix is always smaller than or equal to the determinant of the same matrix with all nondiagonal elements set equal to zero. The equality occurs only for diagonal matrices. Proof: See Appendix. In the derivation of the sensitivity factor in Section 11, no attention has been paid to the correlations which may occur among the Coefficients. The nondiagonal elements of A, were implicitly set equal to zero. Neglecting these correlations can cause a serious overestimation of the error in the roots. Indeed, Lemma 2 tells us that the clas-

5 1054 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 38. NO. 6. DECEMBER 1989 sical approach gives the worst-case estimation of the deviation of the roots when there are correlations. The discrepancy between the predicted errors with the classical approach and the actual errors can be very important, as simulation A in Section IV illustrates. TABLE 1V EMPIRICAL AND APPROXIMATED VALUES OF THE STANDARD DEVIATIONS OF THE ROOTS OF g14 (S) VI. ILLUSTRATIVE EXAMPLES A. Determination of 9 and A, When Some of the Coeficients ak Are Known A Priori Consider the polynomial s2 + 3s + 2. Its roots are - 1 and -2, thus purely real. Suppose the coefficients a1 and a2 are known a priori. So a, is the only parameter occurring in T. Thus (5.8) with k = 1 and k = 2 gives al = K(Re (si) + Re (s2)) E E , E-06 1, E E OE E E E E E E E E so a2 = K. (6.1) K=l Re (s2) = 3 - Re (sl). (6.2) There is only one independent parameter, namely Re ( si ) or Re (s2). Let us for instance choose Re (sl) as independent. Thus 9 = [Re (s,)] and (5.10) becomes (-1)O var{re(s,)} = (-7) var{a,} =var{ao>. \ I After linearization of (6.2) we find that AK=O (6.3) A Re (s2) = -A Re (s,). (6.4) So, the variances of the dependent variables are var {K} = o var {Re (s2)} = var {Re (SI)} = var {a,}. (6.5) To find the covariance matrix of the other variables, it is sometimes easier to apply (5.10) again for another set of independent variables 9, e.g.: B. Comparison of the Empirical and Approximated Standard Deviations of the Roots of g14 (s) Obtained from Simulation A. For simulation [:' 1. A, the vectors AT and A+ are A Im (SI) AT = A9 = 1m(s2) (6.7) A Im (SI41 Aa28 AK Using (5.10), one can find an approximation of the covariance matrix A* of the roots. Table IV contains an estimation of the Cramer-Rao lower bound standard deviation of the roots ( u~iprox. ). The Cramer-Rao matrix of the roots has been found by replacing A+ in (5.10) by the Cramer-Rao lower bound matrix of the coefficients. Simulation A was executed 20 times ( NExp. = 20). The empirical values of the standard deviations (uemp.) are then given by { Irn (si ] with XEmp. { Im (si)} the arithmetic mean of Im (si). Due to the large amount of data (300 spectral lines) processed by our maximum likelihood estimator we can be sure that the estimates of the coefficients have reached their asymptotic values. Statistics then tells us that the ratio uemp. /u~~pmx. has a probability of 90 percent to occur in the interval (0.736, 1.25). Conclusion: From Table IV, it follows that the Cramer-Rao under bound matrix calculated by (5.10) coincides completely with the empirical values. Thus the variances of the root estimates have also reached their Cramer-Rao under bounds. This demonstrates that these roots are the most accurate ones, obtainable from our data and a priori knowledge. So, the computation of the roots did not introduce a supplemental error. The correlations among the coefficients make the ill-condition polynomial much less sensitive to coefficient errors. VII. INFLUENCE OF ROUND OFF ERRORS ON CORRELATEDATA Suppose you have extracted the transfer function of a DUT from measured input and output signals by means of a frequency-domain parameter estimation program. The estimated coefficients of the transfer function will be con-

6 GUILLAUME er d.: ROOTS TO ERRORS IN THE COEFFICIENT OF POLYNOMIALS 1055 taminated by measurement inaccuracies and the noise generated by the DUT itself. To decide how many digits are significant one normally relies on the standard deviation of the estimated coefficients. However, this method can lead to serious waste of information about the correlations. Many more digits usually have to be considered even if these are known to be incorrect. Indeed, rounding off or truncating the coefficients will introduce uncorrelated noise. This additional noise will sum to the diagonal elements of the covariance matrix of the coefficients. Thus the nondiagonal elements will become less meaningful and some information contained in the correlations will be lost. Even small values added to the diagonal elements of A, can cause a significant increase of its determinant and thus also of the determinant of A*. This will result in important perturbations of the roots. Because they are uncorrelated, round off and truncation errors obey the rule of Section I1 and thus can cause an important deterioration in the accuracy of the roots. The sensitivity factor SMAx can be used to determine the minimal number of digits required to maintain enough correlations among the coefficients to obtain roots with an accuracy of for example 1 percent. Let us again consider simulation A. From Fig. 1 it is obvious that all the roots of the estimated polynomial are in the immediate vicinity of the correct roots. Indeed, the estimation algorithm forces the estimated transfer function to fit the measured one. If the position of the poles and zeros is apparent in the measured transfer function, the roots of the estimated polynomials are well defined even if the estimated coefficients are not very accurate. Consider the vector A7, containing the relative error of the unknown coefficients and the vector A+, with the relative error of the independent roots as entries. Then, (5.10) becomes A+,. = F,Ar,. (7.2) The Euclidean norm or 2-norm of A@, is given by To allow only small variations of the roots, (7.3) must be limited: II II 2 < E (7.4) with E a small positive value. Then, (7.4) can be written as ATTQAT, < E ~, Q = FTF,. (7.5) The quadratic form AT :QAT, = E represents an ellipsoid in the N-dimensional AT,-space. All the coefficient combinations represented by a point AT, inside this ellipsoid (see Fig. 3) produce a Aar confined inside a sphere with radius E. The classical approach limits the space of the Atr - Space Fig. 3. Influence of correlated and uncorrelated perturbations on the accuracy of the roots. tolerable errors AT, to approximately a sphere with a diameter equal to the smallest main axis of the ellipsoid. When the positions of the poles and zeros of a transfer function are well defined by the measurements, the estimation algorithm will confine the coefficient errors inside such an ellipsoid. Hence, important correlated perturbations are admitted in the direction of the largest main axis. When the ellipsoid is very elongated (corresponding to strong correlations) it is important to maintain all the estimated coefficients intact. Fig. 3 shows the effect of round off errors. Point 1 in Fig. 3 illustrates the effect of rounding off the correct coefficients, while point 3 shows the impact of round off errors on the estimated coefficients. The probability to have a round off error perturbation in the same direction as the ellipsoids main axis is negligible when the number of estimated coefficients is large. VlII. CONCLUSIONS Frequency-domain parameter estimation techniques can determine accurate poles and zeros, even in the case of high-order systems. Due to the correlations which exist among the coefficients of an estimated transfer function, the sensitivity of the poles and zeros to errors in the coefficients will be significantly decreased. When the result of a measurement process is a set of correlated data, it is no longer admissible to round off the elements of the set in the same way as for single value measurements. The set of data must be considered as an entity. To decide how many decimal digits are relevant one cannot rely on the standard deviation of each element separately. Additional digits have to be considered in order to maintain the information enclosed in the correlations among the elements of the set. Correlated data are more than a set of numbers. They form a closely tied family. If you perturb one of them, the entire family will be harmed. APPENDIX A. Proof of Lemma 1 Lemma 1: F is a regular matrix. Proof: If F is singular then there exists at least one linear relation between the elements of A@, and the entries of A+ will not be linearly independent.

7 1056 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 38, NO. 6. DECEMBER 1989 B. Proof of Lemma 2 Making use of the inequality [4] Lemma 2: The determinant of a positive definite matrix c~~ I c2 ~2... cp is always smaller than, or equal to, the determinant of the same matrix with all nondiagonal elements set equal to PICI + ~ 2 ~ 2 zero. The equality occurs only for diagonal matrices. Proof: The proof we are giving follows the same reasoning as the proof of the theorem of Hadamard [4]. If A = [ AV] is a positive definite matrix and hii > 0 for all i, then the matrix P = [ pij ] with + * + PnCn PI ip2 t.. tp. where equality only occurs if all the c s are the same, and, setting all p s equal to one and substituting ci by hi, we get is also positive definite. Further, we can write that with A = CPCT c = [av], uv = Jx,, ifi =j (B2) = 0, ifi #j. (B3) Therefore, the inequality to prove becomes (C(21PI 5 ( XI2. 034) Thus we have to prove that the determinant of the positive definite matrix P with all entries equal to one on the main diagonal is smaller than or equal to one. Consider the equation \P-hZ( =o (B5 1 with Equation (B5) has n positive roots hi, i = 1, 2,, n, whose sum is equal to the trace of matrix P and whose product is the determinant of the matrix P. with and Thus n A;=JP( i=l n C xi = tr ( P) = n. i= 1 (BIOI For equality, all the roots of (B5) must be equal, which is only possible if P is a diagonal matrix. ACKNOWLEDGMENT The authors wish to express their gratitude to Prof. E. Delbaen of the Vrije Universiteit Brussel in Belgium for the helpful discussions concerning the proof of Lemma 2. REFERENCES [l] A. Ralstone, A First Course in Numerical Analysis. New York: McGraw-Hill, 1965, ch. 8, pp J. H. Wilkinson, The Algebraic Eigenvalue Problem. Oxford, U.K.: Clarendon, [3] H. Vold and C. Y. Shih, On the numerical conditioning of some modal parameter estimation methods, in Proc. 13th Int. Seminar on Modal Analysis, [4] G. H. Hardy, J. E. Littlewood, and G. Pblya, Inequalirks. London, U.K.: Cambridge University, 1973, pp [5] J. Schoukens, R. Pintelon, and J. Renneboog, A maximum likelihood estimator for linear and nonlinear systems-a practical application of estimation techniques in measurement problems, IEEE Trans. Instrum. Meas., vol. 37, pp , Mar