2.6 The optimum filtering solution is defined by the Wiener-Hopf equation

Transcription

1 .6 The optimum filtering solution is defined by the Wiener-opf equation w o p for which the minimum mean-square error equals J min σ d p w o () Combine Eqs. and () into a single relation: σ d p p 1 w o J min 0 Define A σ d p Since p (3) σ d Edn [ ( )d * ( n), p E[ u( n)d * ( n), and E[ u( n)u * ( n), we may rewrite Eq. (3) as A Edn [ ( )d * ( n) Edn [ ( )u ( n) E[ u( n)d ( n) E[ u( n)u ( n) 31

2 E dn ( ) u( n) d * ( n), u ( n) The minimum mean-square error equals J min σ d p w o (4) Eliminate σ d between Eqs. and (4): J( w) J min + p w o p w w p + w w (5) Eliminate p between () and (5): J( w) J min + w o wo w o w w wo + w w (6) where we have used the property. We may rewrite Eq. (6) simply as J( w) J min + ( w w o ) w ( w o ) which clearly show that J(w o ) J min..7 The minimum mean-square error equals J min σ d p 1 p Using the spectral theorem, we may express the correlation matrix as QΛQ M k1 λ k q k q k ence, the inverse of equals 1 M q λ k q k k1 k () 3

3 J r( 0) ( w w w o ) 4.3 (a) There is a single mode with eigenvalue λ 1 r(0), and q 1 1, ence, J( n) J min + λ 1 v 1 ( n) where v 1 (n) q 1 (w o - w(n)) (w o - w(n)) (b) J( n) v λ 1 ( n) ( w o wn ( )) The estimation error e(n) equals en ( ) dn ( ) w ( n)u( n) where d(n) is the desired response, w(n) is the tap-weight vector, and u(n) is the tap-input vector. ence, the gradient of the instantaneous squared error equals ˆ J( n) [ en ( ) w [ en ( )e * ( n) w en ( ) e* ( n) e * ( n) en ( ) w w e * ( n)u( n) u( n)d * ( n) + u( n)u ( n)w( n) 4.5 Consider the approximation to the inverse of the correlation matrix: 1 n ( n+1) µ ( I µ) k k0 where µ is a positive constant bounded in value as 0 < µ < λ max where λ max is the largest eigenvalue of. Note that according to this approximation, we have -1 µi. Correspondingly, we may approximate the optimum Wiener solution as 11

4 w( n+1) 1 ( n+1)p n µ ( I µ) k p k0 n µp + µ ( I µ) k p k1 In the second term, put k i+1 or i k-1: n w( n+1) µp + µ ( I µ) ( I µ) i p k1 µp+ ( I µ)w( n) where, in the second line, we have used the fact that n µ ( I µ) i p i0 w( n) ence, rearranging Eq. : w( n+1) w( n) + µ [ p w( n) which is the standard formula for the steepest descent algorithm J( w( n+1) ) J( w( n) ) --µ g( n) For stability of the steepest-descent algorithm, we therefore require J( w( n+1) ) < J( w( n) ) To satisfy this requirement, the step-size parameter µ should be positive, since µ g( n) > 0. ence, the steepest-descent algorithm becomes unstable when the step-size parameter is negative. 113

5 The corresponding plot of the error performance surface is therefore J(n) r r(0) r(0) r(0) 0 -r/r(0) a (c) The condition on the step-size parameter is 0 < µ < r ( 0) 4.10 The second-order A process u(n) is described by the difference equation un ( ) 0.5u( n-1) + un- ( ) + vn ( ) ence, w 1 0.5, w 1 and the A parameters equal a 1 0.5, a 1 Accordingly, we write the Yule-Walker equations as r( 0) r( 1) r( 1) r( 0) r( 1) r( ) () σ v k0 a k rk ( ) 1 a 0 r( 0) + a 1 r( 1) + a r( ) r( 0) + 0.5r( 1) r( ) (3) Eqs., () and (3) yield 116

6 r(0) 0 r 1 r() -1/ ence, The eigenvalues of are -1, +1. For convergence of the steepest descent algorithm: 0 < µ < λ max where λ max is the largest eigenvalue of the correlation matrix. ence, with λ max 1. 0 < µ < 4.11 un- ( ) 0.5u( n-1) + un ( ) vn ( ) ence, w 1 1 w 0.5 Accordingly, we may write w b r B* as r( 0) r( 1) r( 1) r( 0) r( ) r( 1) σ v k0 a k rk ( ) 1 r( 0) r( 1) 0.5r( ) () r(0) 0 r -/3 Therefore,

7 components subtract coherently, thereby yielding the average power (A /)(1-a). ence, we may express the cost function J as J σ a ν A + σν ( 1 a) M Differentiating J with respect to a and setting the result equal to zero yields the optimum scale factor A a opt A + 4 ( σ ν M) A ( σ ν )( M ) A + ( σ ν )( M ) ( M )SN ( M )SN where SN A σ ν 5.5 The index of performance equals J( w, K ) Ee [ K ( n), K 13,,, The estimation error e(n) equals en ( ) dn ( ) w T ( n)u( n) where d(n) is the desired response, w(n) is the tap-weight vector of the transversal filter, and u(n) is the tap-input vector. In accordance with the multiple linear regression model for d(n), we have T dn ( ) w o ( n)u( n) () where w o is the parameter vector, and v(n) is a white-noise process of zero mean and variance σ v. (a) The instantaneous gradient vector equals 17

8 ˆ ( nk, ) J( w, K ) w [ e K ( n) w Ke K-1 ( n) en ( ) w Ke K-1 ( n)u( n) ence, we may express the new adaptation rule for the estimate of the tap-weight vector as 1 ŵ( n+1) ŵ( n) --µ( ˆ ( nk, )) ŵ( n) + µku( n)e K-1 ( n) (3) (b) Eliminate d(n) between Eqs. and (), with the estimate w(n): ŵ( n) used in place of en ( ) ( w o ŵ( n) ) T u( n) + vn ( ) T ( n)u( n) + vn ( ) u T ( n) ( n) + vn ( ) (4) Subtract w o from both sides of Eq. (3): ( n+1) ( n)-µku( n)e K-1 ( n) (5) For the case when (n) is close to zero (i.e., ŵ( n) is close to w o ), we may use Eq. (4) to write e K-1 ( n) [ u T ( n) ( n) + vn ( ) K-1 v K-1 ( n) 1 u T ( n) ( n) vn ( ) K-1 18

9 v K-1 ( n) 1 ( K-1) ut ( n) ( n) vn ( ) v K-1 ( n) + ( K-1)u T ( n) ( n)v ( K-1) ( n) (6) Substitute Eq. (6) into (5): ( n+1) [ I µk( K-1)v ( K-1) ( n)u( n)u T ( n) ( n)-µkv K-1 u( n) Taking the expectation of both sides of this relation and recognizing that (n) is independent of u(n) by low-pass filtering action of the filter, () u(n) is independent of v(n) by assumption, and (3) u(n) has zero mean, we get E[ ( n+1) { I µk( K-1)Ev [ ( K-1) ( n) }E[ ( n) (7) where E[ u( n)u T ( n) (c) Let QΛQ T (8) where Λ is the diagonal matrix of eigenvalues of, and Q is a matrix whose column vectors equal the associated eigenvectors. ence, substituting Eq. (8) in (7) and using υ( n) Q T E[ ( n) we get υ( n+1) { I µk( K-1)Ev [ ( K-1) ( n) Λ}υ( n) That is, the ith element of this equation is υ i ( n+1) 1 µk( K-1)Ev ( K-1) [ ( n) λ i υi ( n) (9) where υ i ( n) is the ith element of υ( n), and i 1,,, M. Solving the first-order difference equation (9): 19

10 υ i ( n) 1 µk( K-1)Ev ( K-1) n-1 [ ( n) λ i υ i ( 0) where υ i ( 0) is the initial value of υ i ( n). ence, for υ i ( n) to converge, we require that 1 µk( K-1)Ev [ ( K-1) ( n) λ max < 1 where λ max is the largest eigenvalue of. This condition on µ may be rewritten as 0 < µ < K( K-1)λ max Ev [ ( K-1) ( n) (10) When this condition is satisfied, we find that υ i ( ) 0 for all i That is, ( ) 0 and, correspondingly, ŵ( ) w o. (d) For K 1, the results described in Eqs. (3), (7) and (10) reduce as follows, respectively, ŵ( n+1) ŵ( n) + µu( n)en ( ) E[ ( n+1) ( I µ)e[ ( n) 0 < µ < λ max These results are recognized to be the same as those for the conventional LMS algorithm for real-valued data. 5.6 (a) We start with the equation E[ ( n+1) ( I µ) E[ ( n) where ε( n) w o ŵ( n) We note that 130

11 µj min for small µ. (b) From the Lyapunov equation derived in Problem 5.10, we have K 0 ( n) + K 0 ( n) µj min, µ small where only the first term of the summation in the right-hand side of Eq. 8 in the solution to Problem 5.10 is retained. Taking the trace of both sides of this equation, and recognizing that tr[ K 0 ( n) tr[ K 0 ( n) we get for n : tr[ K 0 ( ) µj min tr[ From Eq. (5.90) of the text, J ex ( ) tr[ K 0 ( n) µ --J min tr[ ence, the misadjustment is M J ex ( ) J min µ --tr[ 5.13 The error correlation matrix K(n) equals K( n) E[ ( n) ( n) The trace of K(n) equals tr[ K( n) tr{ E[ ( n) ( n) } E{ tr[ ( n) ( n) } 141

12 Since tr[ ( n) ( n) tr[ ( n) ( n) we may express the trace of K(n) as tr[ K( n) E{ tr[ ( n) ( n) } tr{ E[ ( n) ( n) } The inner product (n) (n) equals the squared norm of (n) which is a scalar. ence tr[ K( n) E[ ( n) From convergence analysis of the LMS algorithm, we have K( n+1) ( I µ)k( n) ( I µ) + µ J min () Initially, (n) is so large that we may justifiably ignore the term µ J min, in which case Eq. () may be approximated as K( n+1) ( I µ)k( n) ( I µ), n small (3) Assuming that σ u I we may further reduce Eq. (3) to K( n+1) ( 1 µσ u ) K( n) Thus, in light of Eq. we may write E[ ( n+1) ( 1 µσ u ) E[ ( n), n small The convergence ratio is therefore approximately 14

13 cn ( ) E[ ( n+1) E[ ( n) ( 1 µσ u ) n w 143