Lecture 4: Dealing with the measurement noise of a sensor

Transcription

1 Barrtz, March - 6, 0 Lecture 4: Dealng wth the measurement nose of a sensor S. Dlhare, D. Mallet, LOMA, Unversté Bordeaux and CNRS, Bordeaux, France E-mal: stefan.dlhare@u-bordeaux.fr LEMA, Unversté de Lorrane and CNRS, Vandoeuvre-lès-Nanc, France E-mal: dens.mallet@unv-lorrane.fr Abstract. In ths lecture, the electronc nose of a measurement sstem and ts phscal orgn s presented, as well as ts man characterstcs (see also lectures L and L5 for temperature measurements. he man propertes of random sgnals (jont probablt denst, expectaton, varance-covarance matrx are also brefl recalled. hese notons are essental for the assessment of resdual error n lnear (see lecture L3 and non-lnear parameter estmaton problems (see lecture L7 and for functon estmaton problems (see lecture L8, Introducton Measurement nose consttutes a crtcal pont that s crucal to understand n parameter estmaton, that s when comparng measurements and a model: the nowledge of nose propertes allows an assessment of the level of the estmaton errors (bas and dsperson n ths tpe of expermental nverse problems. Secton s devoted to the characterzaton of the measurement nose and a to bref descrpton of the dfferent orgns of measurement nose an nverter (that s an expermentalst combned wth a model-bulder can meet. Secton presents the vector-matrx representaton of these characterstcs, for further use n parameter estmaton problems.. Phscs of measurement nose.. Defnton Fgure Random sgnal Lecture : Gettng started page

2 Barrtz, March - 6, 0 A random sgnal (or stochastc process s a sgnal that does not reman unchanged when the experment t stems from s repeated (see Fg.. It s noted (t,ωwhere ω s a test (that s a random varable that contans the outcome of a random draw.. x (t, ω s a realzaton of (t,ωfor a specfc draw ω... Statstcal descrptors rst order descrptor Expectaton = Ensemble mean : m (t = E { (t, ω } (4. It s an ndcator of the average poston of the sgnal at tme t (see Fg. Fgure Realzatons of a random, local probablt denst functon and expectanc varaton Lecture : Gettng started page

3 Barrtz, March - 6, 0 nd order descrptors Instant power = Quadratc mean : P (t=e{ (t, ω } (4. It s a quantfcaton of the average power of the sgnal at a gven tme t Calculaton of the nstant power of sgnal at tme t : P ( t { ( t, } dt t t = E ω (4.3 0 t t + t Varance V (t=e{ l (t,ω m (tl } (4.4 It quantfes the nstant power of random fluctuatons around the mean (dsperson ndcator. Standard devaton σ ( t V ( t (4.5 = It has the same phscal meanng as the varance, but ts phscal unts are the same as the sgnal. he prevous descrptors characterze the sgnal behavor (average poston, dsperson at a tme t. he do not allow for an analss of the relatonshps (dependence that exst between samples. An ndcator s needed for quantfng the level of the «force» that maes a sample at tme t + depend on ts value at tme t. he autocorrelaton functon { ( t ( } R ( t, τ = E t + τ (4.6 It quantfes the correlaton (or the scalar product or the projecton, n the stochastc meanng between ( t + τ and (t (see Fg. 3. Lecture : Gettng started page 3

4 Barrtz, March - 6, 0 Fgure 3 Correlaton between sgnal for a gven tme lag hgh level (left and low level (rght Interpretaton of the autocorrelaton functon: A sgnal ver hghl correlated wth tself exhbts ver slow fluctuatons (wth a smooth appearance, see Fg. 4, top. A sgnal lowl correlated wth tself exhbts ver rapd fluctuatons (wth a chaotc appearance, see Fg. 4, bottom. Fgure 4 pcal slow (top and fast (bottom fluctuatons of sgnal Lecture : Gettng started page 4

5 Barrtz, March - 6, 0 Fgure 5 R ( τ / R ( τ = 0 for a a dscrete whte nose b a thermal nose and c a nose of narrow frequenc nterval.3 Propertes: statonart, ergodct he statonart propert of a sgnal Defnton: A statonart random sgnal s a sgnal whose statstcs do not depend on tme: m ( t = m ; V ( t = V ; P ( t = P (4.7a, b, c Remar : In fact, ths assumpton s nearl compulsor, n order to be able to estmate the characterstcs of the sgnal; Remar : he statonart propert of a random sgnal s analogous to the perodct condton of a determnstc sgnal. In an case, t just an dealzaton. he ergodct propert of a sgnal Defnton: A sgnal s ergodc (n the strong sense, see Fgure 6, f: Lecture : Gettng started page 5

6 Barrtz, March - 6, 0 lm / + / { ( t } ( t, ω dt = E,ω (4.8 Fgure 6 Ergodc sgnal As a partcular consequence, t elds: lm / + ( / t, ω dt = m (4.9 whch means that ts tme average s equal to the ensemble average..4. Examples Dfferent tpes of nose exst: Whte nose: constant spectrum (Fgures 7, 9 and 0 Pn nose: spectrum n /f Brown nose (brownan, sometmes called «red»: spectrum n /f ² Fgure Blue nose: spectrum n f Volet nose: spectrum n f ² Gre nose: blue + pn Lecture : Gettng started page 6

7 Barrtz, March - 6, 0 Fgure 7 Dfferent tpes of nose Other tpes of nose: hermal nose (Johnson Flcer nose Shot nose (Fgure Burst nose Avalanche nose Popcorn nose (see Fgure 8... Fgure 8 Popcorn nose Lecture : Gettng started page 7

8 Barrtz, March - 6, 0 he Popcorn nose (random telegraph sgnal corresponds to a dscrete modulaton of the channel current caused b the capture and emsson of a channel carrer Fgure 9 Whte nose: autocorrelaton (left and frequenc spectrum (rght Fgure 0 Whte nose Lecture : Gettng started page 8

9 Barrtz, March - 6, 0 Fgure Brownan nose (non-statonar Fgure Shot nose Lecture : Gettng started page 9

10 Barrtz, March - 6, 0 Fgure 3 Nose of lmted spectral nterval Fgure 4 Sne sgnal wth random phase.5 Nose n /f a hs tpe of nose les n between a whte nose and a brownan nose : - It s a process wth a long memor. - Man natural sgnals exhbt a spectrum n /f. hs s based on repeated observatons. However no clear theoretcal explanaton s et avalable. It does not stem from an dfferental equaton. - he longest seres of nformaton of data s the level of the Nle rver measured b Egptan over a 3500 ears perod: t follows a law n /f. Lecture : Gettng started page 0

11 Barrtz, March - 6, 0 hs tpe of nose s called b dfferent names : - pn nose (wth a law n /f, - Flcer nose n /f a (n the specal case where a = It s met n electroncs, see Fgures 5 and 6. Fgure 5 Electronc nose - left: spectrum (lnear scale, B. Johnson, Phs. Rev. 6 ( rght: nose spectrum of an amplfer over 3 months (log-log scale, M.A. Caloanndes J. Appl. Phs. 45 ( the heat flux denst q that flows through the wall. Other examples of pn nose are shown n Fgure 6. Flcer nose: - s a generalzaton of the pn nose - ts spectrum s n /f a, wth / < a < 3/ hermal nose of Johnson-Nqust the electronc nose has been observed frst b Johnson n 96 and t has been explaned b hs colleague, Nqust. - t results from the Brownan moton of electrons at constant temperature (thermalzaton. - n ts most general form, t s an elementar stochastc model of nose: whte nose pn nose Brownan nose Lecture : Gettng started page

12 Barrtz, March - 6, 0 Nose n a measurement chan he nose n measurement chan based on a mult-stage measurement chan results from noses at dfferent level of ths sstem, see Fgure 6. Nose for a mult-stage crcut Fgure 6 Pn nose: other examples Nose n a measurement chan he nose n measurement chan based on a mult-stage measurement chan results from noses at dfferent level of ths sstem, see Fgure 7. Fgure 7 - Nose for a mult-stage crcut Lecture : Gettng started page

13 Barrtz, March - 6, 0 Fgure 8 Current and voltage modelng of the nose of an operatonal amplfer At the local level, one has to model the behavor of each stage, see Fgure 8. he correlaton functon and the spectral denst can be used as tools for the characterzaton of the sstem. Fgure 8 Relatonshps between sgnal values at dfferent measurement tmes One has to answer the followng queston: - are an values of f (t at tmes t + t, t + t,..., t m + t, t m + t, t m + + t possble? It s possble to plot the autocorrelaton functon (see Fgure 9: C xx ( τ = lm x ( t x ( t τ dt (4.0 0 Lecture : Gettng started page 3

14 Barrtz, March - 6, 0 he value of ths functon for a zero tme lag s: C xx 0 0 ( 0 = lm x ( t x ( t dt = lm x ( t dt (4. And the sgn of the tme lag does not matter n ts defnton snce ths functon s smmetrc: τ C xx ( τ = lm x ( t x ( t τ dt = lm x ( t' + τ x ( t' dt' = C 0 τ xx (-τ (4. Fgure 9 Example of the shape of an autocorrelaton functon he energ of the sgnal s defned b: e m = lm s ( t dt (4.3 0 It can be decomposed on a frequenc bass, see Fgure 0: e m (n Fourer doman = ( f df (4.4 he quadratc mean n the tme doman s equal to the quadratc value n the frequenc doman. lm x ( t dt = 0 0 ( f df (4.4 Lecture : Gettng started page 4

15 Barrtz, March - 6, 0 Fgure 0 Measurement of the spectral denst. Mathematcal characterzaton of nose for parameter estmaton problems. A remnder of vector random varables.. Statc model and estmaton of the parameters of a probablt law We assume here that the model output s a constant µ (t does not depend on the ndependent varable, called 'tmes' her, whch means that mo = µ. hs means that ts output s 'statc' and repetton of measurements of mo corresponds too a smple random samplng process, wthout replacement here, n order to construct a sample (,,..., m. hs ver classcal procedure s used to get a statstcal estmaton of the stochastc mean µ and/or varance σ of the dstrbuton of (the nfnte number of 'potental' measurements startng from the real dscrete measurements of n the sample. Wth these assumptons, t can be easl shown that, as soon as the number of measurements m s hgh enough, hgher than 30 n practce, the samplee mean Y tends to follow a normal law, noted N here (Central lmt theorem, and the sample varance, noted s S, after a scalng b freedom : σ / m, follows a Ch - law, noted χ here, wth (m degrees of N Y : (, σ / m mo wth : Y = m m = Y and and m S S s s / σ = m m = : χ ( m (Y Y (4.5 hs allows to fnd non-based estmatons of µ and σ, usng the 'hat' notaton (^ to desgnate an estmated value of the correspondng parameter : Lecture : Gettng started page 5

16 Barrtz, March - 6, 0 ŷ mo m = et ˆ σ = s (4.6 m wth and s the observed values of Y and of correspondng measured values. S s, replacng random varables Y b the.. Measurements for a dnamcal model and random vector We consder now a dnamcal model, where measurements of = η ( t, x at m tmes t (for = to m consttute a samplng operaton but now the expectaton of the measurements vares wth tmes. hs means that ths samplng s made for a dnamcal populaton. he frst natural dea s to consder each measurement as the realzaton of a scalar perfect exact random varable Y, of expectaton E ( Y = =η ( t x that s the mo exact, exact t 'perfect' (noseless output of the model whose structure η (,. s exact as well as the set of parameters gathered n a column vector Lecture 3 of ths seres: perfect exact x (see equaton 3. n = + ε (4.7 where ε s the measurement nose, that s the dfference between the measured sgnal and the perfect output of the model that corresponds exactl to ths measurement. Let us note that t s mpossble de to dscrmnate n a measurement the contrbuton of perfect the exact model output from the nose ε. hat s wh the measurement nose s consdered as a random varable. he expectaton of the nose s equal to zero for a good measurement chan, and one sas then that the (drect perfect measurement of s unbased. So, because of (4.7, the expermental sgnal (of expectaton equal to perfect s also a random varable. One calls L the probablt law L followed b nose ε. One good measurng nstrument provdes a nose whose standard devaton σ s constant. hs probablt law s characterzed b several parameters: ts expectaton (that s ts stochastc mean, equal to zero here, ts varance σ as well as possbl other parameters requred to defne the probablt denst functon (pdf f ε of ths random varable. If one lmts oneself to the mean and varance (case of a normal law b example, one notes: perfect ε : L ( 0, σ Y : L (, σ (4.8 A captal character has been used here to desgnate the random varable Y whose Lecture : Gettng started page 6

17 Barrtz, March - 6, 05 realzaton at tme t s the meaured sgnal : so ths measurement s clearl a random varable too. perfect If the expectaton of s dfferent from, ether the model s wrong (case of a model bas, but we have excluded ths assumpton above, or the measurement s based, whch means that n average (that s wth repettons the sensor (wth ts acquston chan and ts calbraton law does not eld the exact value t s supposed to measure. he second dea s to consder the whole set of measurements (a mult-dmensonal sample as a column vector = [ L ] t, that s the realzaton of a vector random varable [ Y Y ] t vector form: where the nose vector ε s also a random vector. Y = L Y m m. So, equaton (4.7 can be gven a = perfect + ε (4.9 In the general case, vector nose ε defned n (4.9 has a probablt law noted L : ( E( ε 0, cov (,... ε : L = ε (4.0 where the (smmetrcal varance-covarance matrx cov ( ε s defned b : cov var ( ε cov ( ε, ε L cov ( ε, ε m var ( ε cov (, ( = cov ( = O ε ε m ε Y (4. O M smétrque var ( ε m Wth ths second pont of vew, the jont probablt denst functon f Y of Y can be defned as followed: Prob ( Y < + d, Y < + d, L, Y < + d n n n fy (,, L, n d d L d n = fy n ( d = d L d n (4...3 Example of a bnormal jont dstrbuton We consder here the partcular case of a Gaussan dstrbuton. hs allows a graphcal plot of the jont pdf f Y, see Fgure, whose expresson s gven below: Lecture : Gettng started page 7

18 Barrtz, March - 6, 05 Lecture : Gettng started page 8 + = = ( ( ( ( ( ( : wth ( ( σ σ ρ σ σ ρ ρ σ σ π x x x x Y, t, t, t, t, Q, Q exp, f mo mo mo mo (4.3 Fgure Bnormal law, case of correlated measurements hree parameters are present n ths law : the varances σ et σ of the two ndvdual random varables Y et Y and ther correlaton coeffcent ρ ( ρ. he varancecovarance matrx of Y s defned n able n the ver general case of a random vector Y of sze and of jont probablt denst functon, ( f Y. hs can be easl generalzed n the case m >.

19 Barrtz, March - 6, 05 able Defnton s and quanttes assocated wth two random varables Expectaton E ( Y = f Y ( dy = mo ( t, x pour = ou Expectaton of a random E ( Y E ( Y = = E ( Y mo mo vector ( t, x ( t, x Margnal dstrbutons (probablt denst functons f Y ( = Prob ( Y < + d = f Y (, d f Y ( = Prob ( Y < + d Specfc case of the bnormal law : = f Y (, d f Y ( = σ exp - π E (Y σ for = or Varances var ( Y = σ = ( E ( Y f Y (, d for = or Covarance cov ( Y, Y = ( E ( Y ( E ( Y = E Correlaton coeffcent ( [ Y E ( Y ] [ Y E ( Y ] = cov ( ε, ε ( Y, Y cov ρ = σ σ f Y (, dy dy Varance covarance matrx var (Y cov ( Y = cov ( Y, Y cov ( Y, Y var (Y σ = ρ σ σ ρ σ σ σ Lecture : Gettng started page 9

20 Barrtz, March - 6, 05 Let us note that, b defnton, the expectaton of a real nose should be equal to zero ( E ( ε = 0. hs vector nose wll be : - non correlated (or ndependent, f ts varance-covarance matrx s dagonal : ε, ε σ cov ( = δ, where δ s Kronecer smbol (null f, and equal to f =, where σ s the varance of ε. - ndependent and dentcall dstrbuted (..d. f ths nose s uncorrelated wth a constant varance : var ( ε = σ. In ths case ts varance-covarance matrx s sphercal : cov (ε = σ I, where I s the dentt matrx of dmensons (m x m. In ths case each component of ε s ndependent and follows the same probablt law.. Propertes of random vectors he noton of random vector has been ntroduced above. In the tpe of applcatons concerned b ths school, t s the column vector of measurements Y, of dmensons (m x. hs vector has an expectaton, noted E (Y, that s a vector of same sze, whose coeffcents are the expectatons of the correspondng coeffcents of Y. It has also a varance-covarance matrx, noted cov (Y, of dmensons (m x m whose coeffcents are the covarances of the coeffcents of Y : [ E ( Y ] = E Y and [ cov ( Y ] = cov (Y, Y ( (4.3 It s alwas possble to lnearl transform sgnal Y usng a lnear transformaton whose determnstc coeffcents are set n a matrx sont rangés G of sze (p x m, n order to get a transformed sgnal Z = G Y of sze (p x. Snce Y s a random vector, such s also the case for vectorel Z. he expectaton and the varance-covarance matrx of Z depend on the same properes of Y: Z = G Y E ( Z = G E ( Y et cov ( Z = G cov ( Y G ( Conclusons In ths short course, the dfferent characterstcs of an electronc nose have been detaled and the resultng effect on the sgnal, that can be consdered as the realzaton of a random vector has been gven. he stochastc propertes of ths random sgnal vector completel depend on the same propertes of the nose vector, f the model used s unbased. So ther nowledge s mportant f one wants to characterze the estmaton error n an expermental nverse problem (see lectures 3, 7 and 8 of ths seres, for example. Lecture : Gettng started page 0