Dept. of Biomed. Eng. BME801: Inverse Problems in Bioengineering Kyung Hee Univ.

Transcription

1 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv Adaptive Filters - Statistical digital sigal processig: i may problems of iterest, the sigals exhibit some iheret variability plus additive oise we use probabilistic laws to model the statistical variability - I adaptive filterig, statistics are ot kow must be iferred from data itself - Fixed filter (FIR): u [ ] y[ ] w - Adaptive filter (FIR): M 1 [ ] w u [ ] k [ ] k0 y wu k u [ ] y [ ] M 1 [ ] w [ ] u [ ] k[ ] [ ] k0 y w u k (1) Idetificatio u [ ] Ukow System d [ ] y [ ] () Iverse modelig 1 Eug Je Woo

2 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv Iput Ukow System u [ ] y [ ] Delay d [ ] - Predictive decovolutio - Adaptive equalizatio (3) Predictio d [ ] Iput Delay u [ ] y [ ] - Liear predictive codig - Spectrum estimatio - Sigal detectio (4) Iterferece cacellatio Sigal Source Primary Sigal d [ ] Noise Source u [ ] Referece Sigal y [ ] - Adaptive oise cacellatio - Echo cacellatio - Adaptive beam formig Eug Je Woo

3 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv Adaptive Filter usig LMS (Least Mea Square) Algorithm Ukow statistics adaptive (learig) algorithm, iterative algorithm Stochastic steepest descet algorithm (1) Formulatio d [ ] y [ ] u [ ] w[ ] + LMS Let [ ] [ ] [ ] [ ] T M w w w w C, u u u M 0 1 M 1 y[ ] w [ ] u [ ] d [ ] y [ ] Defie J ( ) E e[ ] E d[ ] w [ ] u [ ] u[ ] [ ] [ 1] [ 1] T, w o arg mi J( ) M C Sice we do ot kow the statistics, we take the estimate of J(w[]) by its istataeous value as follows J ˆ( ; w [ ]) d [ ] w [ ] u[ ] d [ ] w [ ] u[ ] d [ ] d[ ] u [ ] d [ ] w [ ] u[ ] w [ ] u[ ] u [ ] The goal is to miimize J ˆ( ; w [ ]) by a suitable choice of w[] I LMS algorithm, we use the steepest descet algorithm J ˆ( ; w [ ]) ˆ( ; [ ]) [ ] Jw u u [ ] w [ ] u [ d ] [ ] 3 Eug Je Woo

4 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv 1 w[ 1] Jˆ ( ; ) w u d u u w [ ] [ ] [ ] [ ] [ ] [ ] w u d u w w [ ] [ ] [ ] [ ] [ ] u e [ ] [ ] [ ] () Algorithm Filter output, y[ ] w u [ ] Estimate error, d [ ] y [ ] Update filter, w[ 1] u [ ] e [ ] Repeat (3) Implemetatio Number of taps, M Step size parameter, M 1-0 with tap-iput power = E u [ k] tap-iput power k0-0 with max is the largest eigevalue of R max - 0 tr( R) 4 Eug Je Woo

5 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv Adaptive Filter usig LS (Least Square) Algorithm We approximate esemble average with time average (1) Formulatio d[ ] y[ ] u[ ] w[ ] + e[ ] LS Let [ ] [ ] [ ] [ ] T M w w w w C, u u u M 0 1 M 1 y[ ] w [ ] u [ ], d [ ] y [ ] Defie J ( ) E e[ ] E d[ ] w [ ] u [ ] We take the estimate of J(w[]) by its time average value as follows where N M N Jˆ( ; w [ ]) E ( N ) e [ ] u[ ] [ ] [ 1] [ 1] T, M w o arg mi J( ) M C E For statioary process, ( N ) E N w w o for all We vectorize the sigals as follows Let dmdm dn N Assume d [ ] [ 1] [ ] e the em [ ] em [ 1] en [ ] T Let y ym [ ] ym [ 1] yn [ ] [ M] [ M1] [ N] w u w u w u, y w u[ M] u[ M 1] u[ N] w A with y Aw where y is N 1, w is M 1, A is N rak matrix (ie rak(a) = M) The, y Spa( A ) M Assume A is full 5 Eug Je Woo

6 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv I order to miimize E ( N) e e e e, e where e dy daw, y must be the projectio of d oto Spa( A ) From the orthogoality priciple (OP), aspa( A ) a 0, a Ab ea, 0 Therefore, ( d w A ) Ab 0 Sice b 0, w A A d A A Aw 1 w A A A d 1 A d Fially, y A A A A d This is the least square solutio () Algorithm Costruct the data matrix, A Solve A Aw A d for w Compute y = Aw (3) Implemetatio Number of taps, M Number of data poit, N What if A is rak deficiet? What if N < M? I this case, we have ifiitely may solutios we wat to fid the miimum orm solutio 6 Eug Je Woo

7 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv Adaptive Filter usig RLS (Recursive Least Square) Algorithm Recursive form of LS (least square) algorithm (1) Formulatio di [] yi [] ui [] w[ ] + ei [] RLS We take the estimate of J(w[]) by its time average value of the time iterval 1 i as follows Jˆ( ; w [ ]) E ( ) (, i ) e [ i ] where M Let [ ] [ ] [ ] [ ] T M w w w w C w[] is fixed for time M 1 i Let i u i u i u im i1 u[] [] [ 1] [ 1] T, y[ i] w [ ] u [ i], ei [] di [] yi [] The weightig factor ( i, ) satisfies 0 ( i, ) 1 for i = 1,,, Especially, i the expoetial weightig factor or forgettig factor is defied by ( i, ) where is a positive costat close to 1 but less tha 1 The, i E ( ) e[ i] We defie M M correlatio matrix ( ) at time as i1 i1 i ( ) u() i u () i defie M 1 cross-correlatio vector z ( ) at time as i ( ) () i d () i i1 z u The, by the LS method, the solutio wˆ ( ) of the ormal equatio ( ) wˆ ( ) z ( ) 7 Eug Je Woo

8 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv miimizes E ( ) owever, we wat to recursively compute w ˆ () i for i Note that 1 1 i ( ) u( i) u ( i) u( ) u ( ) ( 1) u( ) u ( ) i1 1 1 i ( ) () i d () i ( ) d ( ) ( 1) ( ) d ( ) i1 z u u z u Therefore, give ( 1) z ( 1), at time, we update ( ) z( ) usig u ( ) d ( ) The, we ca compute w ˆ ( ) from ( ) wˆ ( ) z ( ) The essece of RLS algorithm is to avoid matrix iversio usig the matrix iversio lemma Matrix iversio lemma (or Woodbury's idetity) is as follows Let A B be two positive defiite M M matrices related by 1 1 A B CD C where D is aother positive defiite N M matrix, C is a M N matrix The, The, Let We set up as follows 1 P ( ) ( ) the RLS algorithm is 1 1 A BBC DC BC C B 1 A( ), B ( 1), Cu( ), D ( ) ( 1) 1 1 ( 1) u( ) u ( ) ( 1) 1 P( 1) u( ) k( ) u ( ) ( 1) u( ) 1 1 u ( ) P( 1) u( ) 1 1 P( ) P( 1) k( ) u ( ) P ( 1), the the Riccati equatio for The M M matrix P() is the iverse correlatio matrix M 1 vector k() is the gai vector We also kow ( ) 1 1 ( 1) ( ) ( ) ( ) ( 1) ( ) 1 1 k P u k u P u P( 1) k( ) u ( ) P( 1) u( ) P( ) u( ) 1 ( ) u( ) Now, at time, Therefore, 1 ˆ ( ) ( ) ( ) ( ) ( ) ( ) ( 1) ( ) ( ) d ( ) w z P z P z P u 8 Eug Je Woo

9 Dept of Biomed Eg BME801: Iverse Problems i Bioegieerig Kyug ee Uiv ˆ ( ) ( 1) ( 1) ( ) ( ) ( 1) ( 1) ( ) ( ) ( ) w P z k u P z P u d 1 1 ( 1) z( 1) k( ) u ( ) ( 1) z( 1) P( ) u( ) d ( ) w( 1) k( ) u ( ) wˆ ( 1) P( ) u( ) d ( ) w( 1) k( ) d ( ) u ( ) wˆ ( 1) wˆ ( 1) k( ) ( ) where ( ) is the a priori estimatio error defied by T ( ) d( ) u ( ) w ( 1) d( ) wˆ ( 1) u ( ) A posteriori estimatio error is e ( ) d ( ) wˆ ( ) u ( ) () Algorithm Iitialize the algorithm by P(0) 1 I w(0) 0 for a small positive costat For each time = 1,,, compute 1 P( 1) u( ) k( ) 1 1 u ( ) P( 1) u( ) ( ) d( ) wˆ ( 1) u ( ) w( ) w( 1) k ( ) ( ) P P k u P 1 1 ( ) ( 1) ( ) ( ) ( 1) (3) Implemetatio Number of taps, M Iitializatio Covergece i about M iteratios Sigal distortio 9 Eug Je Woo