Linear Mean-square Estimation. Wednesday, November 30, 11

Transcription

1 Liner Men-squre Estimtion

2 Discrete-time Filtering Let Z(n) desired signl N(n) dditive noise X(n) =Z(n)+N(n) dt/observtions Find the liner men-squre estimtor of Z(n) Ẑ(n) = k= h(n k)x(k) from given set of observtions. Tht is, Min h(n) " = E[Z(n) Ẑ(n)] N(n) Z(n) + X(n) h(n) Ẑ(n) Cusl LTI system 2

3 Remrks The optiml filter is liner system so this is liner MSE problem. The dt used is {X(k), <kpplen} nd the estimtor is this problem is clled filtering. Ẑ(n), thus The prediction problem is when no noise is involved, but we wnt to find n estimte of future vlue of the desired signl Z(n), i.e., the problem is Given Z(n) =X(n) find ˆX(n + ) = h(n k= dt: {X(k), <kpple n} k)x(k) Orthogonlity Principle Error Z(n) Ẑ(n) is orthogonl to the dt. Z(n) Z(n) Ẑ(n) Ẑ(n) plne spnned by dt 3

4 e(n) = Z(n) Ẑ(n) E[e(n)X(k)] = 0 k 2 I Filtering Prediction E[Z(n) Ẑ(n)]X(k) =0, <kpple n E[X(n + ) ˆX(n + )]X(k) =0, <kpple n Norml equtions Filtering E[Z(n)X(k)] = R ZX (n, k) = h(n `)E[X(`)X(k)] <kpplen `= `= h(n `)R XX (`, k) <kpple n If X(n) is wide-sense sttionry, nd X(n),Z(n) re jointly wide-sense sttionry then R ZX (n k) = `= h(n `)R XX (` k) <kpple n 4

5 Prediction E[X(n + )X(k)] = R XX (n +, k) = h(n `)E[X(`)X(k)] <kpplen `= `= h(n `)R XX (`, k) <kpple n If X(n) is wide-sense sttionry, then R XX (n + k) = `= h(n `)R XX (` k) <kpple n Minimum LMS error After the optiml h(n) is found, the minimum error is " min = E[(Z(n) Ẑ(n)) 2 ]=E[(Z(n) Ẑ(n))Z(n)] E[(Z(n) {z Ẑ(n))Ẑ(n)] } =0,orthog. to Ẑ(n) or dt " min = E[Z 2 (n)] E[Ẑ(n)Z(n)] = R X Z(0) h(n k)r ZX (k n) k 5

6 Continuous-time LMSE Estimtion N(t) Z(t) + X(t) h(t) Ẑ(t) Cusl LTI system Assume X(t) nd Z(t) re jointly w.s.s., nd X(t) isw.s.s.: Filtering estimtor Ẑ(t) = Z t h(t )X( )d Orthogonlity principle E pple Z(t) Z t R ZX (t ) = Minimum LMSE error Z t h(t )X( )d X( ) =0 < pplet h(t )R XX ( )d " min = E[(Z(t) Ẑ(t))Z(t)] = R Z (0) = R Z (0) Z t Z t 6 h(t )E[X( )Z(t)]d h(t )R XZ ( t)d

7 Men-squre Clculus 7

8 Let {X(t), t 2 T } be sclr, continuous-time, rndom process such tht E[ X(t) 2 ] < for ll t 2 T Continuity X(t) is continuous t t 0 in the men-squre sense (m.s.s.) i lim E[(X(t) X(t 0 )) 2 ]! 0 t!t 0 Theorem. X(t) is m.s.s. continuous t t 2 T i R X (t, ) is continuous t every point (t, ). Proof E[(X(t + h) X(t)) 2 ] = E[X 2 (t + h)] + E[X 2 (t)] 2E[X(t + h)x(t)] = R X (t + h, t + h)+r X (t, t) 2R X (t, t + h) If R X (.) is continuous t every point when we tke the limit s h! 0we get bove zero. Corollry. If X(t) is sttionry then it is continuous t t 2 T i R X ( ) continuos t = 0. The bove eqution becomes E[(X(t + h) X(t)) 2 ]=2R X (0) 2R X (h) so R X ( ) must be continuous t = 0 for the limit s h! 0 gives zero. 8

9 Corollry 2. If X(t) is m.s.s. continuous t t 0 then m X (t) is continuous t t 0. We hve 0 pple Vr(X(t) X(t 0 )) = E[(X(t) X(t 0 )) 2 ] (E[X(t) X(t 0 )]) 2 so tht E[(X(t) X(t 0 )) 2 ] [m X (t) m X (t 0 )] 2 If X(t) is m.s.s. continuous then lim E[(X(t) t!t 0 X(t 0 )) 2 ]! 0 then lim [m X (t) m X (t 0 )] 2! 0 or m X (t)! m X (t 0 ) t!t 0 Di erentition lim h!0 X(t + h) h X(t) = dx(t) dt Theorem 2. X(t) is m.s.s. di erentible t t 2 T i exists t (t, 2 R(t, Theorem 3. If X(t) is sttionry, X(t) is m.s.s. di erentible t t 2 T 2 R( 2 exists t = 0. 9

10 Corollries Men of Ẋ(t) mẋ(t) =E[Ẋ(t)] = de[x(t)] dt = ṁ X (t) Autocorreltion RẊX (t, ) = E[Ẋ(t)X( )] = E pple lim h!0 X(t + h) h E(X(t + h)x( )) E(X(t)X( )) = lim h!0 h XX(t, X(t) X( ) If X(t) is sttionry, letting t 0 = t then from previous item RẊX (t 0 XX(t 0 ) dt 0 Notice tht in the bove result the di erentition nd E commute so tht de[x(t)x( )] E[Ẋ(t)X( )] = dt XX(t, Following the bove result RẊẊ (t, ) =R[Ẋ(t)Ẋ( )] = d dt d d (E[X(t)X( )]) R XX (t, 0

11 Integrtion X(t) is m.s.s. Riemn integrble over [, b] i lim X(t i ) t i = ti!0 i=0 Z b X(t)dt ti = t i+ t i Theorem X(t) is m.s.s. Riemn integrble over [, b] i R X (t, ) is Riemn integrble over [, b] [, b]. Corollry If X(t) is m.s.s. Riemn integrble over [, b] then Integrtion nd expected vlue commute " Z # b Z b E X(t)dt = E[X(t)]dt = Z b m x (t)dt Double integrtion nd expected vlue commute " Z b Z # b Z b Z b E X(t)dt X( )d = E[X(t)X( )] dtd {z } R X X(t, )

12 Covrince of integrls " Z b Z # d Cov X(t)dt, X( )d c " Z b = E (X(t) = E = " Z b Z b Z d c Z d c! Z!# d E(X(t)))dt (X(t 0 ) E(X(t 0 ))dt 0 (X(t) m X (t))(x(t 0 ) m X (t 0 ))dtdt 0 # E[(X(t) m X (t))(x(t 0 ) {z m X (t 0 ))] dtdt 0 } C X (t,t 0 ) c Conclusion If X(t) is m.s.s. Riemn integrble, the opertions of integrtion nd expectti on commute. Di erentil equtions If X(t) is the input of LTI system represented by di erentil eqution with zero initil conditions n Y (n) (t)+ n Y (n ) (t)+ + 0 Y (t) =X(t) t 0 Desired: sttisticl chrcteristics of the output Y (t) given those of the input X(t) Men clcultion Tking expected vlue of d.e. n (n) Y (t)+ (n ) n Y (t)+ 0 Y (t) = X (t) t 0 Y (t) =E[Y (t)], ( t)=e[(t)] i.e., deterministic d.e. with zero initil conditions. 2

13 Autocorreltion clcultion Consider the opertor d n d n L n,i = n dt n + n i dt n then i L n,2 [Y (t 2 )] = n Y (n) (t 2 )+ n Y (n ) (t 2 )+ + 0 Y (t 2 )=X(t 2 ) X(t )L n,2 [Y (t 2 )] = X(t )X(t 2 ) L n,2 [X(t )Y (t 2 )] = X(t )X(t 2 ) E{L n,2 [X(t )Y (t 2 )]} = E[X(t )X(t 2 )] L n,2 {E[X(t )Y (t 2 )]} = R XX (t,t 2 ) {z } R XY (t,t 2 ) or the di erentil eqution with zero initil conditions: n d n R XY (t,t 2 ) dt n 2 d n R XY (t,t 2 ) + n dt n R XY (t,t 2 )=R XX (t,t 2 ) gin deterministic d.e. tht permits clcultion of R XY (t,t 2 ) given R XX (t,t 2 ). 3

14 If we re interested in R YY (t,t 2 ): L n, [Y (t )] = X(t ) Y (t 2 )L n, [Y (t )] = Y (t 2 )X(t ) E{L n, [Y (t 2 )Y (t )]} = E[Y (t 2 )X(t )] L n, {E[Y (t 2 )Y (t )]} = E[Y (t 2 )X(t )] {z } {z } R YY (t,t 2 ) R XY (t,t 2 ) The initil conditions re obtined by considering for the first cse 0 E[x(t )Y (0)], {z } R XY (t,0) d m nd similrly for the second cse. B )Y (0)] A {z } R XY (t,0), m =,,n dt m 2 4