GTLS ALGORITHMS IN THE FREQUENCY DOMAIN SYSTEM IDENTIFICATION USING NOISE INFORMATION OUT OF A FINITE NUMBER OF REPEATED INDEPENDENT REALIZATIONS

Transcription

1 GTLS ALGORITHMS IN THE REQUENCY DOMAIN SYSTEM IDENTIICATION USING NOISE INORMATION OUT O A INITE NUMBER O REPEATED INDEPENDENT REALIZATIONS Gerd Versteen *, Rik Pintelon Johan Schoukens Vrije Universiteit Brussel (VUB / TW-ELEC), Pleinlaan, B-1050 Brussels, Belgium Telephone: ; ax: gvers@vnet3.vub.ac.be keywords: requency Domain, GTLS, Sample covariance Abstract This paper gives an overview of the stochastic properties of frequency domain total least squares estimators for rational transfer function models of linear time-invariant systems. The generalized total least squares (GTLS) the bootstrapped total least squares (BTLS) methods require the exact knowledge of the noise covariance matrix. The paper studies the asymptotic behavior for the GTLS the BTLS estimator when the covariance matrix is replaced by the sample covariance matrix obtained from a small number of independent repeated observations. The GTLS estimates remain consistent do not loose any asymptotic efficiency when replacing the exact covariance matrix the sample covariance matrix. The BTLS estimator, whose efficiency approaches the maximum likelihood efficiency, needs to be adapted slightly to produce consistent estimates. 1. Introduction Total least squares (TLS) techniques have been applied success in a wide variety of problems [1]. This paper considers its applications in the frequency domain identification of linear time-invariant systems. Known properties of the generalized total least squares (GTLS) the bootstrapped total least squares (BTLS) are presented under the assumption that the noise covariance matrix on the measurements are known []. Based on these results, the GTLS the BTLS estimators are considered in the case where the exact noise covariance matrix is replaced the sample covariance matrix obtained out of independent, repeated observations. It will be shown that the asymptotic properties of the GTLS estimates are not altered by this substitution. The BTLS estimator, on the contrary, needs to be adapted slightly to ensure strong consistency. In order to highlight the main ideas in the analysis of the TLS estimators, only single input single output (SISO) systems are considered. Extensions towards the multi-variable case can be done out loss of generality. This paper is structured as follows: Section describes the asymptotic properties of the weighted generalized total least squares () estimator for deterministic stochastic weighting matrices. It gives the necessary conditions which need to be fulfilled to obtain these asymptotic properties for the frequency domain estimator. Section 3 describes the stochastic framework for the frequency domain identification of linear time-invariant systems. The results of Section are applied in Section 4 on the GTLS BTLS estimator *.This work is supported by the Belgian National und for Scientific Research (NWO), the lemish government (GOA-IMMI), the Belgian government as a part of the Belgian programme on Interuniversity Poles of Attraction (IUAP50) initiated by the Belgian State, Prime Minister s Office, Science Policy Programming. under the assumption that the noise covariance matrix is known. Section 5 studies the asymptotic properties in the case where the noise covariance matrix is replaced by the sample covariance matrix obtained out of a finite number of independent, repeated observations. The theory is illustrated in Section 6 by simulations.. Properties of the This section describes known asymptotic results of the weighted generalized total least squares () estimator for deterministic stochastic weighting matrices. The main goal of this section is to give all the necessary requirements to applique these theorems on the. Whether or not these requirements are fulfilled in the frequency domain will be verified in a later section. Assume that the exact model satisfies A 0 θ 0 = 0, θ 0 R nθ 1 the exact parameter vector, that the measured matrix A m satisfies A m = A 0 + A nθ A 0, A R (1) where A represents a zero mean noisy perturbation on the exact (but unknown) matrix A 0. The solution θˆ is the given by [1], [3] WA ( m Â)C arg min () Â, θˆ subject to Âθˆ = 0 θˆ Tθˆ = 1. θˆ R nθ 1 is the estimated nθ parameter vector,  R the estimate of A 0, W R nθ nθ is a left weighting matrix C R is a square root of the column covariance matrix of WA : T C = E{ A T W T W A}. The matrix C can become singular for certain identification problems. To avoid the post division of A by C, the solution θˆ should be calculated through the quotient singular value decomposition of the matrix pair ( WA, C) [4]. Hence, the global minimizer of () can be computed using a numerically stable non-iterative method. A. Consistency of for deterministic W C. Elimination of  in () gives the equivalent cost function minimized by the estimator θ T T T A L ( θ, Am ) mw WAm θ = θ T C T Cθ (3) subject to θ T θ = 1. To analyze the asymptotic behavior of the estimates, θˆ = arg minl ( θ, Am ), one needs to fulfill the θ following requirements. Requirement 1. All entries of A m are jointly mixing of fourth order, W is deterministic W 1 <, W <, θ T C T Cθ is regular, C is deterministic. Requirement. A 0 the zero mean perturbation A are inde-

2 pendent C T C = E{ A T W T W A} Requirement 3. There exists an exact model A 0 θ 0 = 0. Requirement 4. The excitation is persistent: T T A 0 W WA0 (4) is of rank nθ 1 for ( = included). Uniform (w.r.t. θ ) convergence probability one (w.p. 1) of the cost towards its expected value is guaranteed under requirement 1. Under requirement, the expected value of the cost (3) becomes θ T T T A E{ L ( θ, Am )} 0 W WA0 θ = (5) θ T C T + 1 Cθ The cost (5) is minimal in the exact parameters θ 0 under requirement 3. Requirement 4 guarantees that all the global minima of (5) satisfy A 0 θˆ = 0. Strong consistency of the estimates immediately follows [] a.s. θˆ. (6) lim = θ 0 Convergence in law towards an asymptotic Gaussian distribution can be proven when mixing of order is assumed []. B. Consistency of for stochastic W C. The above results are only valid if both W C are deterministic. In a lot of practical cases W C depend on measured noise covariances /or previously obtained estimates. Example 1. The BTLS estimator, described in section 4.D, adapts W iteratively to increase the efficiency of the estimates. The new weighting is computed using the estimates of the previous iteration. Since these estimates are stochastic, the weighting matrix becomes stochastic too. Example. The statistics of the perturbing noise is often not a priory known. A possible methodology to determine the noise statistics is to perform M independent realizations of A 0 [ k ] A m = A 0 + A [ k ] (7) A [ k ] denotes the value of A corresponding the k th realization (,, M ). or all M> 1, the sample variance 1 M (8) M 1 ( x [ k ] x M )( y [ k ] x M ) H 1 x M (9) M ---- M = x [ k ] gives an unbiased estimate of the exact covariance matrix. Replacing the exact (but unknown) covariances in the by there sample covariance makes the weighting stochastic (see Section 5.C). Asymptotic results for stochastic weighting can be obtained using []. Assume that the cost function L ( θ, A, θ x ) can be written as function of the stochastic parameter vector θ x of finite length nθ x. The estimates are then represented by θˆ ( θ x ) = arg minl ( θ, A, θ x ) (10) θ Besides some continuity conditions [], following requirements need to be fulfilled to obtain consistent estimates. Requirement 5. θ x converges w.p. 1 towards a constant, non-stochastic θ x0 a.s. θ x θ x0. (11) lim = 0 Requirement 6. θˆ ( θ x0 ) is a strongly consistent estimate a.s. θˆ ( θ x0 ). (1) lim = θ 0 Requirement 5 dems that θ x converges sufficiently fast towards θ x0. Strong consistency of the stochastic estimator then follows from requirement 6 []. Although consistency learns us towards what value the estimates converge, it does not tell anything about the asymptotic distribution of the estimates. θ x0 θ x Requirement 7. θ x converges in probability towards θ x0 as O p ( 1 ): θ x θ x0. = ( 1 ) p (13) Requirement 8. or some constant θ x in some neighbourhood of θ x0, the estimates satisfy lim θˆ ( θ θ x ) x θ x0 = 0 (14) Requirement 7 dems that θ x converges sufficiently fast towards such that the stochastic variations of become negligible w.r.t. the Gaussian stochastic variations of the estimates θˆ ( θ x0 ). Requirement 8 puts a restriction on θ x in the sense that the estimates θˆ ( θ x ) converge to some point irrespective of θ x in some neighborhood of θ x0. θ x will only vary the distribution of θˆ ( θ x ) around θ x0. If requirement 5, 6, 7 8 are fulfilled then [] proves that θˆ ( θ x ) converges in law towards the same normal distribution as θˆ ( θ x0 ) if A is mixing of order. 3. SISO requency Domain Identification of Linear Dynamic Systems A. Model equations Consider a linear, time-invariant system out time delay. The (discrete) ourier spectra of the input u 0 () t output signals y 0 () t are related to each other through a rational transfer function real coefficients, i.e. Y 0 ( jω) = G( θω, )U 0 ( jω) where U 0 ( jω) Y 0 ( jω) are the (discrete) ourier transforms of respectively u 0 () t y 0 () t, where G( θω, ) = B( θ B, Ω) q( Ω) θ = B A( θ A, Ω) p( Ω) θ T = θ A T T θ B θa. (15) Ω is a generalized frequency variable, B( θ B, Ω) A( θ A, Ω) are polynomials in Ω respectively order ob oa, q( Ω) p( Ω) describe the polynomial basis functions, θ B, θ A represent the parameter vector of the numerator the denominator. The generalized frequency variable Ω equals jω for continuous time systems, exp( jωt s ) for discrete time systems, tanh( jωτ) for commensurate distribute systems jω for diffusion phenomena. The transfer function model (15) is not identifiable since G( λθ, Ω) = G( θω, ) for any non-zero number λ. Therefore the model parameters θ are constrained by fixing, e.g. a numerator or dominator coefficient or the -norm: θ = 1. In order to avoid that a zero coefficient is set equal to one, the norm constraint is preferred will be used throughout the paper. Using (15), the complex model equation can be written as A( θ A, Ω)Y 0 ( jω) B( θ B, Ω)U 0 ( jω) = 0 (16) for every ω. Mostly the (discrete) ourier spectra U 0 ( jω) Y 0 ( jω) are calculated from the knowledge of N samples of the measured time signals. If the input is periodic an integer number of periods of the steady state response has been measured, then the (dis-

3 The noise covariance matrices of the perturbing noise are in general unknown. A first solution consists in assuming that the perturbing noise can be modeled as filtered white noise. The properties of that noise filter then need to be estimated together parameters of the plant. It has the advantage that is does not require any assumptions on the excitations used. It has the disadvantage that an addi- crete) ourier spectra can be calculated out systematic errors through the discrete ourier transform (DT) on the samples u 0 ( nt s ) y 0 ( nt s ), n = 0, 1,, N 1, T s the sampling period. Model (16) can then be evaluated at the excited DT angular frequencies { ω 1, ω, ω, } ω k { πr ( NT s )}, r = 0, 1,, N. The complex equation (16) is linear in the model parameters can therefore be rewritten as (17) for every ω. The set of real model equations equals oa ob A 0 θ = 0 A 0 R. (18) The identification problem to be solved is: find an θˆ 0 such that (15) is satisfied. B. Stochastic framework In practice the model parameters θ should be estimated using noisy measurements U ( jω k ) Y ( jω k ) m m of the true input output DT spectra ) U 0 ( jω k ) Y 0 ( jω k Y m ( jω k ) = Y 0 ( jω k ) + Y( jω k ) U m ( jω k ) = U 0 ( jω k ) + U( jω k ) (19) Relying on the properties of the DT [5], it is reasonable to make the following assumption. Assumption 1. Ujω ( k ) Yjω ( k ) are zero mean, mixing (over the frequency), jointly correlated, complex distributed rom variables, independent of U 0 ( jω k ) Y 0 ( jω k ), satisfying E 4. requency Domain Estimators Known Noise Covariance (0) (1) A. The Maximum Likelihood Estimator The maximum likelihood (ML) cost function calculated for zero mean, independent (over the frequency), jointly correlated, complex normal distributed rom variables Ujω ( k ) Yjω ( k ) [6] equals Re( Y 0 ( jω)q( Ω) T ) Re( U 0 ( jω)p( Ω) T ) Im( Y 0 ( jω)q( Ω) T ) Im( U 0 ( jω)p( Ω) T ) Yjω ( k ) Ujω ( k ) Yjω ( k ) E Yjω ( Ujω ( k ) k ) Ujω ( k ) = 0 Y ( jω k ) U ( jω k ) = θ A = 0 θ B σ Y ( jωk ) σ YU ( jω k ) σ UY ( jω k ) σ U ( jωk ) ML L ( θ, Um, Y m ) = e( θ, jω k ) e( θ, jω k ) = w ML ( θ, jω k )εθ (, jω k ) () (3) w ML ( θ, jω k ) = Re( A( θ A, Ω k )B ( θ B, Ω k )σ YU ( jω k )) (4) A( θ A, Ω k ) σ Y ( jωk ) B( θ B, Ω k ) + + σ U ( jωk ) One of the drawbacks of the ML estimator is that it requires an iterative minimization procedure that it can not be guaranteed whether the computed minimum is the global one. B. Weighted Generalized Total Least Squares or a deterministic weighting W = diag( [ wjω ( 1 ), w( jω 1 ), w( jω ),, wjω ( ), w( jω )]) wjω ( k ) 0 finite, the cost becomes L θ Um w ( jω k ) εθ (, jω k ) (,, Y m ) = (5) w ( jω k )w ML ( θ, jω k ) subjected to θ T θ = 1. Calculating the expected value of (5) gives E{ L ( θ, Um, Y m )} = L ( θ, U0, Y 0 ) + 1 (6) for every deterministic weighting. Since the expected value of the cost function is minimal in the true parameter values ( L ( θ, U0, Y 0 ) = 0), the estimates are strongly consistent under assumptions described above. C. The Generalized Total Least Squares The Generalized Total Least Squares (GTLS) is a special case of the in the sense that the weighting matrix W equals the identify matrix. D. The Bootstrapped Total Least Squares Adding an appropriate frequency-dependent weighting to the estimator is the key solution to improve its efficiency. Comparing the cost function (5) the maximum likelihood (ML) cost () learns that the optimal weighting equals wjω ( k ) = w ML ( θ, jω k ). θ is unfortunately unknown, so that only an approximation of the optimal weighting can be calculated through an initial guess θˆ of the model parameters. The weighting matrix W BTLS becomes diag( [ wml ( θˆ, jω1 ), w ML ( θˆ, jω 1 ),, w ML ( θˆ, jω )]) (7) This gives the one-step BTLS estimates [6]. The one-step BTLS estimator is strongly consistent for deterministic θˆ. Due to the appropriate frequency weighting the estimates have nearly ML efficiency [6]. The efficiency can be improved further on by using θˆ = θˆ BTLS to calculate an improved weighting in (7), recalculating the BTLS estimates so on until convergence is obtained. The multi-step BTLS estimator is strongly consistent in the absence of modeling errors []. This can easily be seen if the estimates of the previous BTLS step, θˆ BTLS, is considered as the stochastic parameter θ x in section.b. Requirement 5 is fulfilled since θˆ BTLS is a strongly consistent estimate obtained from the previous iteration while requirement 6 is satisfied since the is strongly consistent for all deterministic θ x, θ x = θ 0 included. Remark that the multi-step BTLS should start a deterministic or strongly consistent estimate of θ 0. This can be done using e.g. a the GTLS estimator or a estimator a deterministic left weighting. The multi-step BTLS estimates also converges towards the same distribution as L ( θ, Um, Y m, θ 0 ). This can easily be seen by veri- BTLS fying requirement 7 8. Remark that requirement 8 can only be guaranteed in the absence of modeling errors. 5. requency Domain Estimators using Sample Covariance

4 tional model, therefore model selection, is required. Consistency can even be lost when choosing an incorrect noise model if both input output of the plant are perturbed. This is e.g. the case for linear systems under feedback. A non-parametric noise model provides a good alternative for the parametric noise model. It assumes that M> 1 independent realizations U m ( jωk ) Y m ( jωk ) (,, M ) are available of [ k ] [ k ] U 0 ( jω k ) Y 0 ( jω k ). This dems e.g. repeated measurements of the steady state response [ k ] Y m ( jωk ) = Y 0 ( jω k ) + Y [ k ] ( jω k ). (8) [ k ] U m ( jωk ) = U 0 ( jω k ) + U [ k ] ( jω k ) The question then arises: what are the asymptotic properties ( ) of the frequency domain estimators described above when replacing the exact covariance matrix the sample covariance matrix σˆ Y ( jωk ) σˆ YU ( jω k ). (9) σˆ UY ( jω k ) σˆ U ( jωk ) A. The Sample Maximum Likelihood Estimator When replacing the exact covariance matrix the sample covariance matrix in the ML formulation (), it losses the property that it is a maximum likelihood estimator. Therefore, the estimator will be called the sample maximum likelihood (SML) estimator. SML L ( θ, Um, Y m ) = e( θ, jω k ) (30) e( θ, jω k ) = w SML ( θ, jω k )εθ (, jω k ) (31) w SML ( θ, jω k ) = Re( A( θ A, Ω k )B ( θ B, Ω k )σˆ YU ( jω k )) (3) A( θ A, Ω k ) σˆ Y ( jωk ) B( θ B, Ω k ) + + σˆ U ( jωk ) Reference [7] performs a complete study of the asymptotic properties the loss in efficiency of the SML. Under the assumption that the perturbing noise is normal distributed independent over the frequencies, strong consistency of the SML estimates has been proven for M 6 while the covariance matrix on the estimates satisfies M C θ, SML = C. (33) M 3 θ, ML Hence, the SML has a loss in efficiency as will be seen in the examples. B. The Sample Weighted Generalized Total Least Squares The Sample Weighted Generalized Total Least Squares S (S) L ( θ, Um, Y m ) equal to w ( jω k ) εθ (, jω k ) (34) w ( jω k )w SML ( θ, jω k ) assumes a deterministic weighting w ( jω k ). Consider the denominator written as θ T C T Cθ. Under the assumption that the perturbations are mixing of order 4 it can easily be proven that the square matrix C T C (of finite dimension) converges w.p.1 towards its expected value []. The mixing of order 4 guarantees that the sample covariances calculated are mixing of order. The strong law of large numbers [] implies the strong convergence for M> 1. Section then implies that the S estimates θˆ S are strongly consistent. The estimates converge in law towards the same asymptotic distribution as the estimates θˆ if mixing of order is assumed. As a result, the Sample GTLS (SGTLS), W the identity matrix, is strongly consistent too. Remark 1. Besides the mixing condition, no assumptions are required concerning the statistics of the noise as function of the frequency. No noise model should even exist. C. The Sample Bootstrapped Total Least Squares The BTLS estimator which uses the exact covariance matrix has the nice property that it produces a consistent estimate during each iteration that its global minimum can be computed using a numerically stable non-iterative method. urthermore, its efficiency approaches the ML efficiency. Replacing the exact covariance matrix by the sample covariance matrix in the BTLS estimator - denoted by SBTLS - violates the assumptions that, for finite M, the number of the stochastic parameters nθ x must be finite since both the left the right weighting matrices W C are function of the sample covariance matrices. The results of Section.B are applicable to the S estimator since only C T C has a finite number of element. The BTLS estimators has, however, a stochastic W whose number of elements tends to infinity as the number of frequencies increases. Consistency can be re-established by adapting the weighting W such that it is modeled using a finite number of parameters. This can be done by estimating a parametric model for the noise covariance matrix. This parametric model must be parametrized using finite number of parameters should not be equal to the underlying noise model. No underlying noise model should even exist. If the noise covariance matrix can be represented by a parametric model whose parameters are estimated in a consistent way, then consistency is guaranteed. The type of parametrization has, however, an impact on the efficiency of the estimates. Example 3. Assume that the input output noise are generated by a white noise source, passing through a stable, linear time-invariant filter. The noise spectra can then be modeled using a rational approximation in the frequency domain. Example 4. Assume that the exact noise covariance σ ( jω) is sufficiently smooth over ω. Projecting the sample covariance σˆ ( jω) on a finite dimensional space then overcomes the problem that the number of stochastic parameters tends to infinity. In the rest of the paper we will concentrate the approach of example 4. Assumption. Consider the matrix B R nθ x, the basis functions of a nθ x dimensional space nθ x fixed finites B T B is regular for ( = included). urthermore it is assumed that all elements of B are finite. Theorem 1. Consider σ = [ σ ( jω 1 ), σ, ( jω )] T, σˆ = [ σˆ ( jω 1 ), σˆ, ( jω )] T B satisfying assumption. Consistency of the BTLS is re-established when the weighting W is calculated using BB ( T B) 1 B T σˆ Proof. Under assumption, it can easily be seen using the definition of the 1-norm that B T 1 is bounded. This implies that B T σˆ is mixing of order if σˆ is mixing of order. σˆ is mixing of order since the perturbing noise is mixing of order 4. Using the law of large number for mixing signals, it is known that B T σˆ converges w.p. 1 towards its expected value []. The regularity of B T B implies that ( B T B) 1 B T σˆ can be chosen as the stochastic vector θ x which converges w.p. 1 towards its expected value ( B T B) 1 B T σ. The projection of the parameter space of θ x on the parameter space of the

5 variances then results in BB ( T B) 1 B T σˆ. Remark. The parametric noise model for the noise covariance matrix is extracted off-line, i.e. before the estimation itself is performed. Hence, its asymptotic properties are independent of the estimator used later on. Remark 3. The parametric noise model should only be used to calculate the weighting matrix W. The right weighting matrix C must be calculated the original sample covariances σ to obtain consistent estimates. In the simulations, choices for B are considered. 1. B = [ 1,, 1] T which will be denoted by SBTLS1. I.e. the left weighting W SBTLS1 is determined using the sample mean (over ) of the sample variances (over M). This works well in practice if the perturbing noise is white noise.. If the perturbing noise is colored, then SBTLS1 will poorly approximate the ML weighting. Therefore, a more appropriate basis needs to be used to described the perturbing noise spectrum. This can be done using e.g. Gaussian basis functions to compute W SBTLS : ( B [ nm, ] e ω n µ m ) ( ζ m ) =. (35) The location of the centre of the Gaussian functions µ there m variance ζ m must be determined experimentally to obtain the highest efficiency possible. The consistency of the estimates is, however, independent of the choice of µ m ζ m. These two cases will be compared the SBTLS estimates where W uses the non-parametrized weighting w SML ( θˆ, jω k ). This estimator will be denoted by SBTLS0. 6. Simulation Examples A. Description The exact input U 0 ( jω) is chosen equal to 1 for all considered frequencies. The exact output spectrum Y 0 ( jω) satisfies G 0 ( jω)u 0 ( jω) ob G 0 () s α k s k oa =. k = 0 β k s k k = 0 (36) The simulated device under test is a fifth-order Butterworth filter an extra transmission zero at ω = 3 rad/s. The coefficients of the transfer function are given in Table I. U [ k ( jω) is chosen to be zero β 0 β 1 β β 3 β 4 β e- 1.1e-3 5.0e-5 α 0 α 1 α 1 0 1/9 Table I. Coefficients of the transfer function of the fifth-order Butterworth filter a transmission zero. mean complex Gaussian noise variance equal to M 100. Y [ k ] ( jω) is a zero mean complex Gaussian noise source a variance of M 100 passing through a fourth order Butterworth lowpass filter its 3dB point at 1 Hz. ig. 1 represents the amplitude plot of Y 0 ( jω) the variance of the perturbing noise on the sample mean of Y [ k ] ( jω). The frequency grid is chosen as a set of 50 equally distributed frequencies in the b of [ 5 50, 5] Hz. One hundred disturbed data sets are generated for M= 34610,,,,. or each set the model parameters are estimated using the 1. GTLS: the GTLS the exact covariance matrix.. SGTLS: the GTLS the sample covariance matrix. 3. BTLS: the BTLS the exact covariance matrix.. (db) req. (Hz) ig. 1. Amplitude characteristic of the fifth order Butterworth filter a transmission zero (full line), together the spectrum of the output noise (dashed line). 4. SBTLS0: the SBTLS the sample covariance matrix, out parametrizing the sample covariance matrix to compute W. 5. SBTLS1: the SBTLS the sample covariance matrix, using the sample mean (over ) of the sample covariance matrix (over M ) to compute W. 6. SBTLS: the SBTLS the sample covariance matrix, using the Gaussian basis function approximation of the sample covariance matrix to compute W. The number of basis functions equals 1 µ m equi-spaced ζ m equal to ML: the ML estimator the exact covariance matrix. 8. SML: the ML estimator the sample covariance matrix. B. Simulation Results ig. represent the mean square error (MSE), expressed in db, on MSE (db) req. (Hz) ig.. MSE over 100 realizations for M =. full line: SBTLS, ML, BTLS Cramér-Rao bound; dotted line: SBTLS1; dash-dot: SBTLS0; dash-dot-dot: GTLS SGTLS; dash-dash: SML. the estimated transfer functions MSE( ω) = E{ Ĝ( jω) G 0 ( jω) } SBTLS SBTLS1 SBTLS0 GTLS SML (37) obtained using the 100 independent simulations. The difference in

6 mean square error of the SBTLS, ML, BTLS estimates the Cramér-Rao bound (CR) is not visible on the graphical resolution of the plot. The same can be said about the mean square error of the SGTLS the GTLS. ollowing conclusions can be draw using ig.. 1. Although that the GTLS the SGTLS are both strongly consistent, there efficiency is very poor, i.e. the MSE is much larger than the Cramér-Rao lower bound.. The MSE of the ML the BTLS estimates - the exact noise covariance - are close to the Cramér-Rao bound. The efficiency of the SML SBTLS0 decreases more than 10 db for M =. Even consistency for the SML can not be guaranteed for [7]. Remark that the MSE of the SML the SBTLS0 M = estimators follow each other closely. 3. The MSE for high frequencies of the SBTLS1 estimates is large w.r.t. the Cramér-Rao bound. This is a consequence of taking the sample mean (over ) of the sample variances (over M). It is as if the left weighting is computed for output noise a flat spectrum. The fact that the exact output noise spectrum has more than 50dB dynamic range explains the loss in efficiency. Consider the mean loss efficiency w.r.t. the Cramér-Rao lower bound to study the efficiency of the different estimators as a function of M. 1 MSE( jω -- k ) CR( jω k ) (38) Table II represents (38) for the different estimators for different M ML SML SBTLS SGTLS Table II. Mean loss in efficiency (38) for different estimators. numbers of realizations, M. Using this table, one can conclude that 1. the ML - exact covariance matrix - approaches the CR lower bound.. the SML - sample covariance matrix - losses a lot of efficiency for M the SBTLS0 has approximately the same loss in efficiency as the SML. 4. the SBTLS1 produces strongly consistent estimates but is inefficient. The loss in efficiency is approximately independent of M. 5. the SBTLS approaches the Cramér-Rao bound very closely. 6. the SGTLS, although strongly consistent, is inefficient for all M. C. Important Remark The efficiency of the SBTLS depends heavily on the parametrization used for the noise spectra. If the spectra of the perturbing noise are very smooth, then it is easy to approximate the noise spectra using a simple parametric model. If, however, the noise filter of the exact noise model has poles high quality factors, then the problem of finding a good noise model using a small number of repeated realizations can only be solved when the number of spectral lines around that resonance frequency is sufficiently large. Hence, the number of frequencies must be large to obtain efficient estimates. 7. Conclusions This paper first sets up a general framework for proving asymptotic properties for the where both weighting matrices may depend on some stochastic variable. The use of this framework is illustrated using the GTLS the BTLS for frequency domain identification. A major drawback of these identification schemes is that they assume that the non-parametric noise models are known. In practice, the noise models are obtained using e.g. the sample covariance from repeated, independent realizations M> 1. The paper demonstrates that the asymptotic properties of the GTLS are not changed by replacing the exact covariance matrix the sample covariance matrix. Unfortunately, the GTLS estimates are inefficient. Therefore the BTLS was introduced in the literature. When replacing the exact covariance matrix the sample covariance matrix in the BTLS estimator, its estimates become inconsistent. The paper describes how the left weighting of the formulation needs to be adapted to obtain consistent estimates. The performances of the SBTLS the SML estimators are compared (both using the sample covariance matrix). In the example used, the SBTLS estimator outperforms the SML estimator even when the noise covariance matrix is obtained using 10 independent realizations. The number of realizations below which the SBTLS outperforms the SML depends strongly on the perturbing noise spectra on the parametrization used for the left weighting matrix. It can be concluded that the SBTLS is a good alternative for the SML when only a small number of repeated observations are available ( M< 6) when the spectra of the perturbing noise are sufficiently smooth. The SBTLS - contrary to the SML - also produces consistent estimates for M> 1 does not dem that the perturbing noise is normally distributed. References [1] Van Huffel S. J. Vewalle, The Total Least Squares Problem: Computational Aspects Analysis, Volume 9 of the rontiers in Applied Mathematics, SIAM, Philadelphia, [] Versteen G., H. Van hamme R. Pintelon, General framework for asymptotic properties of generalized weighted nonlinear least squares estimators deterministic stochastic weighting, IEEE Trans. Autom. Contr., No. 10, pp , [3] Golub G.H. C.. Van Loan, Matrix Computations, nd ed., Johns Hopkins University Press, Baltimore, MD, [4] C. C. Paige. Computing the Generalized Singular Value Decomposition. SIAM J. Sci. Stat. Comput., Vol. 7, no. 4, pp , [5] Brillinger D.R., Time Series: Data Analysis Theory (exped edition), Holt, Rinehart Winston, Inc., New York, [6] Pintelon R., P. Guillaume, Y. Rolain, J. Schoukens, H. Van hamme, Parametric Identification of Transfer unctions in requency Domain, a Survey., IEEE Trans. Autom. Contr., Vol. AC-39, No. 11, 1994, pp [7] Schoukens J., R Pintelon, G. Versteen, P. Guillaume, requency Domain System Identification using non-parametric noise models estimated from a small number of data sets, Automatica, 33, No. 7, 1997, To be published