Detection and Estimation Theory

Transcription

1 ESE 524 Detetion and Estimation heory Joseh A. O Sullivan Samuel C. Sahs Professor Eletroni Systems and Signals Researh Laboratory Eletrial and Systems Engineering Washington University 2 Urbauer Hall (Lynda answers) jao@wustl.edu J. A. O'S. ESE 524, Leture 4, 3/3/9

2 Linear Estimation x and y are jointly Gaussian. Problem : Find the exeted value of x given y. Jointly Gaussian Posterior is Gaussian and MSE estimate = MAP estimate x μx Kxx Kxy N, =, μ N y y Kyx Kyy E xy ( μ K) μ n+ m ln, ln ((2 π det( ) ( μ ) ( μ ) Kxx Kxy x x ( x y) = ) K) x 2 2 x y y K μ yx Kyy y y Kxx Kxy x μx x ln ( xy, ) = [ I ] = μ Kyx Kyy y y J. A. O'S. ESE 524, Leture 4, 3/3/9 3

3 Linear Estimation Solution for MMSE estimate uses the blo matrix inversion exression Result is simle, easily interretable, fundamental Kxx Kxy x μx x ln ( xy, ) = [ I ] = μ Kyx Kyy y y ( ) ( ) Κ xx K Κ xx K xyκ yyk yx Κ xx K xyκ yykyx KxyΚ xy yy = K yx Κ yy Κ yyk yx ( Κ xx K xyκ yyk yx ) Κyy + ΚyyK yx ( Κ xx K xyκ yykyx ) KxyΚ yy Κ μ ( xx K xyκ yy ) ( ) x x K = yx Κ xx K xyκ yykyx KxyΚ yy μ y y ˆ MMSE ( μ ) = μx + xy yy y x K Κ y J. A. O'S. ESE 524, Leture 4, 3/3/9 4

4 Linear Estimation Udated ( ) xˆ MMSE = μx + KxyΚ yy y μy Posterior mean Prior mean Attenuated by unertainty New information Amlified by orrelation J. A. O'S. ESE 524, Leture 4, 3/3/9 5

5 Linear Estimation Problem 2: Assume that the mean vetors and joint ovariane matrix for X and Y are nown. Among all linear estimates of X as a funtion of Y, find the one that minimizes MSE Assume x and y are zero mean random variables with nown joint ovariane matrix. Find the linear estimator that minimizes the trae of the error ovariane matrix ( xay)( xay) minr E A = minr Κ xx KxyA AKyx + AΚ yya A ( ) ( ) = minr Κ K Κ K + AK Κ Κ AK Κ A xx xy yy yx xy yy yy xy yy = r Κ K Κ K, with minimum at A= K Κ xx xy yy yx xy yy J. A. O'S. ESE 524, Leture 4, 3/3/9 6

6 Linear Estimation Problem : x and y are jointly Gaussian. Find the exeted value of x given y. Problem 2: x and Y have nown seond order statistis. Among all linear estimates of x as a funtion of y, find the one that minimizes MSE Answer 2 = Answer Fundamental (Orthogonality) Proerty: he error in the estimate t is orthogonal to the variables used in the estimate. Error ovariane matrix ( x [ x y] ) x [ x y] E E E = Κ K Κ K xx xy yy yx ( ) ( [ ] ) = [ ] ( ) ( [ ] [ ]) E E E E E x x y y x x y y y = E E E x y x y y = J. A. O'S. ESE 524, Leture 4, 3/3/9 7

7 Reursive Linear Estimation Dt Data Model and Problem Statementst t Suose a zero-mean, stationary Gaussian random roess (GRP) with nown ovariane funtion is given. Problem : Find the minimum mean square error estimate of the resent value of the GRP given the revious values. a: Derive the result as a transversal filter and derive the order-reursive udates (from to +). b: Derive the result as a lattie filter and derive the order- reursive udates for the oeffiients (refletion arameters). Problem 2: Assume that the GRP satisfies an autoregressive (AR) model of order. 2a: Find the maximum lielihood estimates of the AR arameters, inluding the time-reursive and order-reursive udates. 2b: Find the time- and order-reursive reursive udates for the lattie filter oeffiients (refletion arameters). J. A. O'S. ESE 524, Leture 4, 3/3/9

8 Linear Predition heory [ ] GRP Jointly Gaussian Er [ n] =, E rr n nl = l Distributions for any E [ rn rn, rn2,... rn ] = wr n+ w2rn wrn subset of random variables Jointly Gaussian linear r( n ) = [ rn... rn2 rn ] estimation results aly; w = [ w w... w ] estimate of urrent value is a linear ombination of Er [ n rn, rn2,... rn] = wr( n) revious values r( n ) Linear ombination defines E ( n ) rn r r = + = a transversal filter n Imlementation through a ( ) =, [ ] i, j ij =... taed delay line w =, w = Stationary ovariane 2 matrix is oelitz, E ( rn wr( n ) ) = oeffiients in a transversal filter are indeendent of time J. A. O'S. ESE 524, Leture 4, 3/3/9

9 Linear Predition heory Matrix artition leads to udates of inverse Estimate is rossovariane vetor times the inverse of the ovariane matrix of the observations times the observations Error variane equation [ ] Er [ ] =, E rr = n n nl l E [ r r, r,... r ] = wr + w r w r r n n n2 n n 2 n2 n = [ 2 ] [ w w... w ] ( n ) r... r r n n n w = lays an imortant role ( ) Er [ r, r,... r ] = wr( n) n n n2 n r( n ) E ( n ) rn r r = + = n =, =... w i, j [ ] ij =, w = ( wr( ) ) E r n 2 n = J. A. O'S. ESE 524, Leture 4, 3/3/9 2

10 Linear Predition heory Covariane matrix is oelitz oelitz onstant diagonals Order reursion derives from artition of ovariane matrix by order. here are two standard artitions. he seond uses an exhange matrix J that has ones along the antidiagonal. Somewhat loose on subsrits J = J. A. O'S. ESE 524, Leture 4, 3/3/9 r ( n ) E ( n ) r = r r n =, =... n + ( ) [ ] i, j ij w = = + = ( ) J + = J + =

11 Linear Predition heory Equations resulting from the orthogonality roerty are the normal equations erminology: forward redition error of order ([ r ] r w n ) E ( n) r ( n ) = w =, w = a w = w a = = + w + = J. A. O'S. ESE 524, Leture 4, 3/3/9 4

12 Order Udate on Inverse: Ran One Udate ( ) ( ) } ( ) ( ) } rows + = = row row ( ) } J + = J } rows ( ) J J ( ) ( ) ( J J ) ( J ) + = ( ( ) ) ( ( ) ) J J J + J J J ( J) + ( ) ( ) J = ( ) ( ) J + J J J J =, JJ = I, J = J J. A. O'S. ESE 524, Leture 4, 3/3/9 5

13 Order Udate on Inverse: Ran One Udate ( ) ( ) ( ) ( ) + + = = = + a ( ) a ( ) ( ) J ( J ) + = = J J ( ) + J ( ) J = ( ) + b b b = = Ja Jw J. A. O'S. ESE 524, Leture 4, 3/3/9 6

14 Baward Predition i Predit r n- from following values Baward redition error of order ; baward redition error filter Exhange matrix omes in again Same error variane as in forward redition Basis for order udate: udate baward and forward redition error oeffiients ([ r ] n r w ) E ( n) r ( n) = J w =, w = J = J = Jw b = Jw J + b = = J Jw + = Jw b + J + 2 = Jw J + + J + = J Jw = + + Jw Δ = b J. A. O'S. ESE 524, Leture 4, 3/3/9 + 7

15 Order r Udate For order udate, ombine equations to anel to and bottom terms b + 2 = + Jw = w a + J + + Jw = J w = + Δ = Ja = b b+ =+ 2, a = + b Δ + + 2, + = = a Δ b + 2, + = = a + J. A. O'S. ESE 524, Leture 4, 3/3/9 Δ Δ 2 2 8

16 Order r Udate Order udate requires multilies li to find Δ One division + multilies to get the udate wo multilies to get the next error variane otal omlexity: 2 +3 Sum from to is ( +)+3 his is the transversal filter version of linear redition. Initialization: =, a = b =, =, = = 2. Udate refletion oeffiient and error variane 2 Δ Δ = + Ja, + = 2 = a 3. Udate redition error filters b Δ b + = a Δ b a + = a 4. Reursion ste = + = +, +, return to ste 2 J. A. O'S. ESE 524, Leture 4, 3/3/9 9

17 Lattie Filter Define filter in terms of Δ Define the forward and baward redition errors F ( n) = a r ( n), G ( n) = b r ( n), + + Order udate equations give Δ b F+ ( n) = a+ r+ 2( n) = + 2( n) r a Δ b = r+ 2 ( n ) 2 ( n ) r + a Δ = F( n) G( n) b Δ G+ ( n) = b+ r+ 2( n) = + 2( n) r a Δ = G( n) F( n) Δ F + ( n) F ( n ) G ( n) = + G( n ) Δ J. A. O'S. ESE 524, Leture 4, 3/3/9 2

18 Dt Data Driven Both data udates and order udates Data udates: ran one udate to a matrix is a ran one udate to its inverse ( n) ( N) ( N) r r ( N) ( N) N r( n ) r n = + ( N) = n= + n w ( N) = ( N) ( N) N n= + ( ) 2 = rn w ( N) r( n ) ( N) ( N) ( N) ( N) ( N) + ( N + ) = + ( N ) + ( N ) r+ ( N ) r ( ) r r+ N + ( N) r + ( N) + ( N ) + ( N ) J. A. O'S. ESE 524, Leture 4, 3/3/9 2