Adaptive Filters. un [ ] yn [ ] w. yn n wun k. - Adaptive filter (FIR): yn n n w nun k. (1) Identification. Unknown System + (2) Inverse modeling

Transcription

1 Adaptive Filters - Statistical digital signal processing: in many problems of interest, the signals exhibit some inherent variability plus additive noise we use probabilistic laws to model the statistical variability - In adaptive filtering, statistics are not known must be inferred from data itself - Fixed filter (FIR): un [ ] yn [ ] w - Adaptive filter (FIR): M [ ] = w u [ ] = k [ ] k= 0 yn n wun k un [ ] yn [ ] M [ ] = w [ ] u [ ] = k[ ] [ ] k= 0 yn n n w nun k () Identification un [ ] Unknown System yn [ ] () Inverse modeling - - BME, KU

2 Input Unknown System un [ ] yn [ ] Delay - Predictive deconvolution - Adaptive equalization (3) Prediction Input Delay un [ ] yn [ ] - Linear predictive coding - Spectrum estimation - Signal detection (4) Interference cancellation Signal Source Primary Signal Noise Source un [ ] Reference Signal yn [ ] - Adaptive noise cancellation - Echo cancellation - Adaptive beam forming - - BME, KU

3 Adaptive Filter using LMS (Least Mean Square) Algorithm Unknown statistics adaptive (learning) algorithm, iterative algorithm Stochastic steepest descent algorithm () Formulation yn [ ] un [ ] w[ n ] LMS Let [ ] [ [ ] [ ] [ ]] T M w n = w n w n w n C, n = [ u n u n u n M ] 0 M yn [ ] = w [ n] u [ n] = yn [ ] Define { } { } J ( ) = E e[ n] = E d[ n] w [ n] u [ n] u[ ] [ ] [ ] [ ] T, w o = arg min J ( ) M C Since we do not know the statistics, we take the estimate of J(w[n]) by its instantaneous value as follows Jn ˆ( ; w [ n ]) = ( d[ n] w [ n] u[ n] )( d[ n] w [ n] u[ n] ) = = d [ n] d[ n] u [ n] d [ n] w [ n] u[ n] w [ n] u[ n] u [ n] The goal is to minimize J ˆ( n; w [ n]) by a suitable choice of w[n] In LMS algorithm, we use the steepest descent algorithm Jn ˆ( ; w [ n ]) ˆ( ; [ ]) [ ] J n w n = = u n u [ n ] w [ n ] u [ n ] d [ n ] ( ) BME, KU

4 w[ n ] = µ Jˆ ( n; ) ( ) ( ) = µ u[ n] d [ n] u[ n] u [ n] = µ u[ n] d [ n] u [ n] = w n µ u n e n [ ] [ ] [ ] () Algorithm Filter output, yn [ ] = w u [ n] Estimate error, = yn [ ] Update filter, w[ n ] = µ u [ n] e [ n] Repeat (3) Implementation Number of taps, M Step size parameter, µ M - 0 < µ < with tap-input power = E{ un [ k] } tap-input power k= 0-0 < µ < with λ max is the largest eigenvalue of R λ max - 0 < µ < tr( R) BME, KU

5 Adaptive Filter using LS (Least Square) Algorithm We approximate ensemble average with time average () Formulation d[ n] y[ n] u[ n] w[ n ] e[ n] LS Let [ ] [ [ ] [ ] [ ]] T M w n = w n w n w n C, n = [ u n u n u n M ] 0 M yn [ ] = w [ n] u [ n], = yn [ ] Define { } { } J ( ) = E e[ n] = E d[ n] w [ n] u [ n] We take the estimate of J(w[n]) by its time average value as follows where N M N Jˆ( n ; w [ n ]) E ( N ) e [ n ] = = u[ ] [ ] [ ] [ ] T, n= M w o = arg min J ( ) M C E > For stationary process, ( N ) E{ N } w[ n ] = w = w o for all n We vectorize the signals as follows Let = [ dmdm dn] e = em [ ] em [ ] en [ ] T Let N Assume d [ ] [ ] [ ] then [ ] y = ym [ ] ym [ ] yn [ ] = [ M] [ M ] [ N] w u w u w u, [ [ M] [ M ] [ N] ] y = w u u u = w A with y = Aw where y is N, w is M, A is N rank matrix (ie rank(a) = M) Then, y Span( A ) M Assume A is full BME, KU

6 In order to minimize E ( N ) = e e= e = e, e where e= d y = daw, y must be the projection of d onto Span( A ) From the orthogonality principle (OP), a Span( A ) a 0, a= Ab ea, = 0 Therefore, ( d w A ) Ab = 0 Since b 0, w A A = d A A Aw = A d Finally, ( ) w = A A A d ( ) y = A A A A d This is the least square solution () Algorithm Construct the data matrix, A Solve A Aw = A d for w Compute y = Aw (3) Implementation Number of taps, M Number of data point, N What if A is rank deficient? What if N < M? In this case, we have infinitely many solutions we want to find the minimum norm solution BME, KU

7 Adaptive Filter using RLS (Recursive Least Square) Algorithm Recursive form of LS (least square) algorithm () Formulation di [] yi [] ui [] w[ n ] ei [] RLS We take the estimate of J(w[n]) by its time average value of the time interval i n as follows n Jˆ( n ; w [ n ]) E ( n ) β ( n, i ) e [ i ] = = where n> M Let [ ] [ [ ] [ ] [ ]] T M w n = w n w n w n C w[n] is fixed for time 0 M i n Let i = [ u i u i u i M ] u[] [] [ ] [ ] T, yi [] = w [ n] u [] i, ei [] = di [] yi [] The weighting factor β ( ni, ) satisfies 0 < β ( ni, ) for i =,,, n Especially, n i the exponential weighting factor or forgetting factor is defined by β ( ni, ) = λ where λ is a positive constant close to but less than Then, n n i E ( n) = λ e[ i] We define M M correlation matrix Φ ( n) at time n as n λ Φ = u u n i ( n) ( i) ( i) define M cross-correlation vector z ( n) at time n as n n i ( n) = λ () i d () i z u Then, by the LS method, the solution wˆ ( n) of the normal equation Φ ( n) wˆ ( n) = z ( n) minimizes E ( n) owever, we want to recursively compute w ˆ () i for i n Note that BME, KU

8 ( n) n n i λ λ u() i u () i u( n) u ( n) λ ( n ) u( n) u ( n) Φ = = Φ n n i ( n) = λ λ ( i) d ( i) ( n) d ( n) = λ ( n ) ( n) d ( n) z u u z u Therefore, given Φ( n ) z ( n ), at time n, we update Φ ( n) z( n) using u ( n) d ( n ) Then, we can compute w ˆ ( n) from Φ ( n) wˆ ( n) = z ( n) The essence of RLS algorithm is to avoid matrix inversion using the matrix inversion lemma Matrix inversion lemma (or Woodbury's identity) is as follows Let A B be two positive definite M M matrices related by A= B CD C where D is another positive definite N M matrix, C is an M N matrix Then, Then, Let We set up as follows P ( n) =Φ ( n) the RLS algorithm is ( ) A = B BC D C BC C B A=Φ ( n), B = λφ( n ), C= u( n), D = λ Φ ( n) u( n) u ( n) Φ ( n) λ u ( n) Φ ( n) u( n) λ P( n) u( n) k( n) =, then the Riccati equation for λ u ( n) P( n ) u( n) Φ ( n) = λ Φ ( n) P( n) = λ P( n) λ k( n) u ( n) P ( n) The M M matrix P(n) is the inverse correlation matrix M vector k(n) is the gain vector We also know k P u k u P u = λ P( n) λ k( n) u ( n) P( n) u( n) = P( n) u( n) =Φ ( n) u( n) ( n) = λ ( n) ( n) λ ( n) ( n) ( n) ( n) Now, at time n, Therefore, ˆ ( n) ( n) ( n) ( n) ( n) λ ( n) ( n ) ( n) ( n) d ( n) w =Φ z = P z = P z P u BME, KU

9 wˆ ( n) = P( n) z( n) k( n) u ( n) P( n) z( n ) P( n) u( n) d ( n) =Φ ( n) z( n) k( n) u ( n) Φ ( n) z( n ) P( n) u( n) d ( n) = w( n) k( n) u ( n) wˆ ( n ) P( n) u( n) d ( n) = w( n ) k( n) d ( n) u ( n) wˆ ( n) = wˆ ( n ) k( n) ξ ( n) where ξ ( n) is the a priori estimation error defined by T ξ ( n) = d( n) u ( n) w ( n ) = d( n) wˆ ( n) u ( n) A posteriori estimation error is en ( ) = dn ( ) wˆ ( n) u ( n) () Algorithm Initialize the algorithm by For each time n =,,, compute λ P( n) u( n) k( n) = λ u ( n) P( n ) u( n) ξ ( n) = d( n) wˆ ( n) u ( n) w( n) = w( n ) k ( n) ξ ( n) P P k u P P(0) = δ I w(0) = 0 for a small positive constant δ ( n) = λ ( n) λ ( n) ( n) ( n) (3) Implementation Number of taps, M Initialization Convergence in about M iterations Signal distortion BME, KU