CORRELATION TEST OF RESIDUAL ERRORS IN FREQUENCY DOMAIN SYSTEM IDENTIFICATION

Transcription

1 IAC Symposium on System Identification, SYSID 006 Marc , Newcastle, Australia CORRELATION TEST O RESIDUAL ERRORS IN REQUENCY DOMAIN SYSTEM IDENTIICATION István Kollár *, Ri Pintelon **, Joan Scouens ** * Budapest University of Tecnology and Economics Department of Measurement and Information Systems H-5 Budapest, Magyar tudóso rt.. Hungary ax: , ollar@mit.bme.u ** Vrije Universiteit Brussel, Dienst ELEC, Belgium Pleinlaan, B-050 Brussel, Belgium, fax: [Joan.Scouens,Ri.Pintelon]@vub.ac.be Abstract: Residual errors (deviations between measurements and system models) can be caused by several reasons: observation/process noise, nonlinear products, system transients, unmodelled dynamics, etc. Te first two cannot be explained by linear models, te latter two can. Identification procedures can be stopped wen te latter are not present in a reasonable size model. Terefore, we need to distinguis among tese error sources. Tis paper analyses components of te residual errors, and suggests a simple automatic metod to effectively suppress system transients and unmodelled dynamics during iance analysis. Tis allows robust noise analysis, and ten reliable model validation. Te metod is based on different properties of te error sequences in te frequency domain, witout te need of separate observation noise analysis or of full-blown nonlinear analysis. Copyrigt 006 IAC Keywords: system identification, residual errors, nonlinear errors, correlation test, uncorrelatedness, witeness, parameter estimation, frequency domain. INTRODUCTION Model validation in system identification usually involves testing of te residuals: te difference between measurements and te system model. Wen te model properly describes system beavior, te residuals contain only observation/process noise. In tis case, te properties of te residuals coincide wit te teoretically expected properties. If tests verify tis assumption, we consider te model validated, at least according to tis test. In practice, residual errors comprise not only observation/process noise, but also unmodelled dynamics, unexplained system transients, and/or nonlinear beavior of te system. Terefore, care must be taen wen analyzing measured data for noise: unmodelled dynamics and/or unexplained system transients will yield erroneously large error levels, wit support of te lemis-hungarian bilateral agreement B-/04, and OTKA TS49743

2 leading to validation of models wit large and oterwise unacceptable errors. On te oter and, nonlinear products cannot be modelled by a linear model, so tey need to be incorporated into te error level to be tested, especially wen te experiment is designed to ave te nonlinearities act lie noise. Te noise contributions, and te nonlinearity products wit excitation aving some randomness (Gaussian noise or random pase excitation), form uncorrelated sequences in te frequency domain, wile non-modeled dynamics and system transients yield strongly correlated sequences. Based on tis, a igpass filtering operation can be used to remove te latter. Tis idea will be explored and executed in te paper. DEINITIONS Te usual framewor applied in frequency domain system identification is as follows (Pintelon and Scouens, 00). ig. Te framewor for frequency domain system identification (a) in time domain (b) in frequency domain (ourier coefficients) Te unnown linear system is excited by u 0 (t), and te dynamic response is y 0 (t). Bot input and output signals are measured, wit observation noises n u (t) and n y (t), respectively. or te sae of simple analysis, we assume ere tat te excitation is periodic, we measure in steady-state, and sampling is coerent (DT of te time sequence contains armonic components at just te corresponding lines). Note tat tese assumptions are not necessary, but mae our considerations and discussion simpler. At a given frequency f, we can write tat H ( f ) ( f ) Y0 ( f ) U 0 B. () A In teory, te residual at f can be determined by subtracting te transfer function from te ratio of te measured ourier amplitudes: ( f ) Y ( f ) ( f ) Rt, () U A or, in a form linear in te system coefficients: t B ( f ) A( f ) Y B( f ) U. (3) In practice, te transfer function coefficients are not nown, we only estimate tem. Terefore, te practically calculable residual series is: ( f ) Aˆ ( f ) Y Bˆ ( f ) U, (4) wic is, for good estimates of te coefficients of B and A, close to (3). Wen te input and output observation noises are normally distributed, are independent, and additive, tese residual values are complex-valued, normally distributed, ave circular symmetry, wit expected value zero, and iance { t( f )} A( f ) σ Y + B( f ) Re cov{ U, Y } B( f ) A( f ) σ U, (5) ( ) E{ U U Y Y } were cov{, Y } ( )( ) U 0 0. Te iance of te practically calculable sequence (4) is somewat different, since in it, estimated quantities are involved. Similarly to te correction necessary for empirical iances (see Bendat and Piersol, 986, Eq. (.7)), te iances are sligtly modified: p { ( f )} σ { ( f )} (6) n / At different frequencies, tese random iables are uncorrelated. In oter words, te sequence ( ) f along te frequency axis is uncorrelated. Te iances of te individual samples cange wit frequency, tus for simple testing, we usually normalize tem, and investigate te sequence ( f ) Aˆ ( f ) Y Bˆ ( f ) U. (7) std{ ( f )} f t { ( )} Tese are, in teory, iid complex, circularly symmetric Gaussian iables, wit iance. It is easy to construct tests for tis, for example by cecing te percentage of te absolute values of (7) being under or anoter preselected value. Tis all corresponds to teory, and wors well in simulations. However, wen woring wit measured data, te validation step often fails, even if te fit of te transfer function to te frequency response is seemingly very good. Excellent signal-to-noise

3 ratio can easily result in poor validation results te better SNR, te poorer te validation. Tis seems to be puzzling, so it deserves consideration and requires improvement of te test. Wen looing into te cause, one can quicly find tat te reason is tat te terms used in te validity test, sown in (7), ave too large absolute values. Te deviations between measurement and model are too large wit respect to te noise we ave in te model. 3 ANALYSIS O THE RESIDUALS In order to analyze te cause, we need to examine te factors wic yield te residuals. In general, te residuals of a fitted model ((), (3), or (4)) are caused by te following factors: observation noise, nonlinearity products, unmodelled dynamics (e.g. undermodeling), transients during measurement. loose witout nowing muc about te system, we cannot design an optimal filter. Tere is owever one more aspect wic elps in te specification. Since we do not want to lose too muc of te lengt of te sequence we ave, we migt say tat we need a possibly sort imp ulse response. rom time domain signal processing, we now tat from 3-coefficient armonic windows, te one wit best suppression at zero frequency is te sine wave wic smootly touces te frequency axis, te Hanning window. Terefore we will apply te following IR filter (a ind of inverse Hanning window) to te frequency domain sequence: [ ( ), (0), () ] [ 0.5,, 0.5] (8) We may notice tat tis is noting else tan a double differentiator in te discrete domain. Te transfer function is illustrated in ig.. Note tat tis is in te inverse transform domain of te frequency-domain residual sequence. Te first two are deviations wic cannot be mo d- eled by a linear system. Teir frequency domain sample sequences are uncorrelated (Pintelon and Scouens, 00). We wis tat te test be not sensitive to tem. Te second two components are related to poor modeling, tus we want to detect tem in validation, and eliminate tem by better models. Since tey are usually described in frequency domain by low-order rational forms, teir samples are igly correlated. If we want to test te magnitudes of te residual samples, we need to select levels wic correspond to te first two factors, but wose determination is insensitive to te last two factors. A igly correlated sequence as lowpass nature, wile te uncorrelated one is wite (uncorrelated), tat is, as constant spectrum. Terefore, discrimination is possible troug ig-pass filtering of te sequence. We need to apply a igpass filter te frequency domain sequence, ten use te result to estimate te RMS value of te first two components. Tis will be more precisely described in te following section. 4 SELECTING A ILTER In te frequency domain, we do filtering on te residual sequence. Te specification is still quite ig. Transfer function of te Hanning window It can be seen tat below 0., suppression is better tan 0 db. or longer impulse responses, even better suppression could be acieved, but we do not need tis in practice. or a reasonable estimate of te standard deviation of te noise, tis will be OK. We do not need a very precise value anyow, only a good guess. By using tis, te major part of te unmodelled dynamics and transients will be removed, and tis is enoug to mae validation sensitive to suc errors, since tese are in excess to te measured RMS value. 3

4 Te RMS value of te filtered sequence is different from te RMS value for te non-filtered one. or statistical tests, we need te latter. It is straigtforward to calculate tat application of (8) increases te iance of an input wite noise by a factor of Terefore, RMS filtered RMS. (9).5 5 THE SUGGESTED BASIC ALGORITHM According to te above tougts, an algoritm for robust validation is as follows:. calculate te frequency domain residual sequence as in (4),. apply igpass filtering as in (8), 3. determine te RMS value from te filtered sequence, 4. correct te RMS value to remove te effect of filtering as in (9) 5. calculate te autocorrelation from te residual sequence, and apply statistical test for te presence of unmodeled dynamics and transients. 6 DETAILS AND EXTENSIONS 6. Statistical Properties of te Residual Sequence Let us assume tat in te residual sequence taen in te frequency domain (see (4)), no transients or unmodelled dynamics are present, tat is, (n) contains only complex, approximately uncorrelated noise. Tis is teoretically not perfectly uncorrelated (removal of te estimated model introduces some coiance), but tis effect is very small, so we may assume an almost uncorrelated sequence, wit zero mean and iance (6). We use filter (8) on tis sequence to remove te effect of transients and unmodelled dynamics, tat is, to produce a filtered signal wic contains prima rily information on te noise and nonlinear products. 6. Te Effect of te inite Number of Samples in te Determination of te Standard Deviations We normalize te sequence by dividing eac sample by its standard deviations. Tis sounds very good, but in practice, we do not ave access to te precise values of tese standard deviations, so we approximate tem by teir estimates from a finite number of samples. or finite sample numbers, tis sligtly increases te iance of te normalized samples by te following factor (Pintelon and Scouens, 00): std M, (0) M { ( n) } { ( n) } M M / { R ( m) } std R ( m) M { } () M M 5/3 were e M is te sample normalized by te standard deviation calculated from M samples, and R is te correlation. or validity tests, te uncorrelatedness of te normalized sequence is tested. or tis purpose, te RMS of te normalized sequence is determined from te sequence itself, so tis increase is automatically taen into account. 6.3 Te Effect of Higpass iltering We suppress lowpass contents by applying filtering wit (8) as *. Te filtered sequence is more or less free from unmodelled dynamics and transients. Its first moments are, assuming noise contents only E { ( n) } 0, { ( )} ( ) 5 ( ) σ. σ σ n () Terefore te iance of te initial residual sequence can be estimated as Validation Test.5 ( ) ( ) (3) We want to test te residual sequence for uncorrelatedness. or tis, we calculate te autocorrelation of te original sequence as Rˆ m m ( m) ( ) ( + m) (4) If te residual sequence is uncorrelated, te expected value is close to zero for m? 0. More precisely, te expected value at zero is equal to te iance of te sequence, { Rˆ ( 0) } { ( ) } ˆ E, (5) σ and te expected value at oter lags is muc smaller tan tis: E m { ˆ R ( m) } E ( ) ( + m) << E Rˆ ( 0) m { } (6) or completeness, we can determine te iance of te correlation estimate: 4

5 { ˆ R ( 0) } ( 0), (7) m C but for testing purposes, we need te iance of te lag values: { Rˆ ( m > 0) } Rˆ ( 0) wit ( ) 0 σ 4 { } C ( 0) σ m m (8) C being te autocoiance at zero lag (see Bendat and Piersol, 986). rom tis, te standard deviation is std { Rˆ ( m > 0) } C ( 0) σ m ( ) m.5 m ( ) (9) tat ( ) Validation tests can be executed by evaluating te correlation of te residual sequence, and cecing if at nonzero lags it beaves lie correlation of a sequence of zero-mean of random iables, wit standard deviation as in (9). or tis, consider also Rˆ m > 0 is zero mean, approximately complex Gaussian wit circular symmetry, as a consequence of te central limit teorem. Terefore, ( m ) ˆ > 0 R is exponential, wit { ˆ ( 0) } λ / R m >, and tis can be tested for. ˆ m > 0 values above te tresold Percent of ( ) R ( - P) stdrˆ { } - ln : Percent Bound std { Rˆ } std { Rˆ } std { Rˆ } A plot of te values and te limits gives direct information for te user. 6.5 requency Dependent Scaling In te above considerations, we tacitly assumed tat te normalized residual sequence is stationary: its iance is constant. However, since te beavior of te sequence may be different at different frequencies (colored noise), it maes sense to apply frequency dependent normalization. Tis can elp us to ave a constant-iance normalized residual sequence, even if te iance of te residual sequence (4) depends on te frequency (te time domain total noise is colored). Te teoretically correct way of doing tis is to apply nown standard deviation values. However, especially wit te above approac, calculating te combined effect of noise and nonlinearities, tese are a priori unnown. We need to determine te values from te sequence itself. But, for eac frequency, we ave one residual value only, wic does not allow to mae a reasonable estimate of te iance. Terefore, we cose a compromise. Wen te number of residuals is sufficiently large, we evaluate (3) for segments. Tat is, (3) is used for < 3. or 3 40, we apply 0 ˆ σ ( i) ( i + ), (0) i,,..., 0 Last, wen 403, we use ˆ ( i) σ ( i) ( i / / / ( i) ( / + ), i,..., / ), i ( i + ) +,,..., / / / +,... () Te rule seems to be a bit complicated. Te rationale is tat for large values of, we cose to average about values, and decreasing, we want to provide smoot transition to te average of all values for < 3. 7 IS NONLINEAR ANALYSIS AN ALTERNATIVE? We would lie to ave a descriptor wic accounts for nonlinearity and noise, but not for unmodelled dynamics. Tis is provided by te results of nonparametric nonlinear R analysis (Pintelon and Scouens, 00; Scouens et al, 005; Kollár et al, 006). By using tese tecniques, validation of linear models will give correct results: indicate unmodelled dynamics in measurements using te best linear model, but will not be sensitive to nonlinearities, since tese are incorporated into te total iance. Tis sounds very good, but may be applied wit some care. Nonlinear analysis as presented in te above papers is somewat signal dependent. Terefore, its results are perfectly applicable only wen te excitation signal is similar in te nonlinear iance analysis and in te parametric system identification steps (random pase multisine wit te same spectrum). Moreover, wile te robust metod (Kollár et al, 006) correctly estimates te total iance, te non-robust one can sligtly over- or underestimate it. Also, experiments for te evalua-, 5

6 tion of nonlinearity limits are somewat more laborious to execute tan for simple linear-model identification. 8 EXAMPLE Our practical example is linear analysis of a robot arm (te excitation is torque, te response is displacement at te oter end). Te cost function is about 30 times te teoretical value, tat is, te RMS of te residuals is 30 times larger tan expected from te residuals. Tis is not very large, but according to validation, te model is not acceptable. Applying te above algoritm, te correlation test (Pintelon and Scouens, 00) validates te model properly (ig. ). Te very same total iances can also be used to test te magnitude of te residuals (ig 3). Again, te total iances yield model validation. 9 SUMMARY An algoritm as been presented wic discriminates between noise and nonlinear errors on te one side, and unmodeled dynamics and transients on te oter side. Using tis, model validation of a linear model becomes robust wit respect to nonlinear errors. REERENCES ig. Validation of te linear model by correlation test. Te continuous line represents te 95% probability bound of te time domain lag correlation, based on te total iance (tis paper). Te dotted line is te 95% bound of linear analysis tis does not validate te model. ig. 3 Validation of te linear model. Te continuous line represents te 95% probability bound based on te total iance (tis paper), te dotted line te 95% bound of linear analysis. Te metod based on total noise iance validates te model, te one based on linear iance does not. J. S. Bendat and A. G. Piersol (986), Random Data: Analysis and Measurement Procedures, Jon Wiley and Sons. I. Kollár, R. Pintelon, and J. Scouens (006), requency Domain System Identification Toolbox for Matlab: Caracterizing Nonlinear Errors of Linear Models. IAC Symposium on System Identification, SYSID 006, Mar. 006, Newcastle. Software Demonstration Session. R. Pintelon, and J. Scouens (00). System Identification. A requency Domain Approac. IEEE Press, Piscataway, NJ. R. Pintelon and J. Scouens (00), Measurement and Modeling of Linear Systems in te Presence of Non-Linear Distortions, Mecanical Systems and Signal Processing 6(5), pp J. Scouens, R. Pintelon, T. Dobrowieci and Y. Rolain (005), Identification of linear systems wit nonlinear distortions, Automatica, Volume 4, Issue 3, Marc 005, pages G. Simon, J. Scouens, and Y. Rolain (000), Automatic Model Selection for Linear Time- Iniant Systems, Proceedings of tet IAC Symposium on System Identification, SYSID 000, Santa Barbara, CA, USA, -3 June 000, Vol. I., pp Extended electronic version: Simon, G., J. Scouens, and Y. Rolain, Automatic Model Selection for Linear Time-Iniant Systems Practical Issues ttp:// 6