EE 225a Digital Signal Processing Supplementary Material

Transcription

1 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL EE 225a Digital Signal Processing Supplementary Material. Allpass Sequences A sequence h a ( n) ote that this is true iff which in turn is true iff which in turn is true iff is said to be allpass if it has a DTFT that satisfies H a ( e jω ). () H a ( e jω jω )H a ( e ) h a ( n)h a ( n) δ( n) H a H a (4) For rational Z transforms, the latter relation implies that poles (and zeros) of z H a (2) (3) are cancelled by zeros (and poles) of H a In other words, if H a has a pole at z c, then H a ---- must z have have a zero at the same z c, in order for (4) to be possible. ote further that a zero of H a ---- at z c is a zero of H a at z -. Consequently, (4) implies that poles (and zeros) z c z

2 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 2 have zeros (and poles) in conjugate-reciprocal locations, as shown below: c /c Intuitively, since the DTFT is the Z transform evaluated on the unit circle, the effect on the magnitude due to any pole will be canceled by the effect on the magnitude of the corresponding zero. Although allpass implies conjugate-reciprocal pole-zero pairs, the converse is not necessarily true unless the appropriate constant multiplier is selected to get unity magnitude. Consider the Z transform of the form This has a pole at c and a zero at / c, as shown in the above figure. It is easy to verify that it is allpass, so an allpass Z transform may consist of the product of any number of such terms. Any non-unity multiplier on (5) would prevent this from being allpass, by the definition in (). ote further that a pure delay term z M for M > 0, which is obviously allpass, has M poles at zero and M zeros at infinity, so it automatically satisfies the allpass property. So does a pure advance term z M for M > 0. A transfer function of the form is a product of a pure delay or advance term and terms of form (5), and hence is allpass. Conversely, in order to have pole-zero pairs at conjugate reciprocal locations, and unity magnitude on the unit circle, any rational allpass Z transform must have form (6). This can also be written where H a z c (5) cz H a () z z M z + + a z a a z a z H a Az ( ) z M A - z Az ( ) + a z a z. (8) (6) (7)

3 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 3 Evaluating this at z e j ω it is easy to directly verify that (7) is allpass. ote that there is no need for the allpass filter to be causal for it to be stable. It can have poles at infinity, with corresponding zeros at zero. It can also have poles anywhere outside the unit circle, as long as there is a corresponding zero at the reciprocal-conjugate location. 2. Minimum-Phase Sequences Definition: The sequence h m ( n) with rational Z transform H m is minimum phase if it:. is stable, 2. is causal, and 3. has all zeros inside or on the unit circle. Be careful with zeros at infinity, as introduced for example by an uncanceled term, which mae a sequence non-minimum phase. The sequence is strictly minimum phase if it is minimum phase and also has no zeros on the unit circle. An LTI system is said to be minimum phase if its impulse response is a minimum-phase sequence. Importance: A strictly minimum-phase system has a stable, causal, and minimum-phase inverse. This is easily seen by observing that the poles of the inverse are zeros of the system, and zeros of the inverse are poles of the system. Fact: Any stable rational Z transform and an allpass Z transform, Hz () can be factored into a cascade of a minimum-phase Hz ( ) H a H m. (9) Proof is by construction: Locate the poles and zeros of Hz (). Assign all those inside the unit or on the unit circle to H m. For each zero (or pole) outside the unit circle at location c, assign to H m a zero (or pole) at /c. Then assign a pole (or zero), to cancel this zero (or pole), at /c z

4 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 4 to H a, and a zero (or pole) at c to mae the filter allpass. In pictures, Hz ( ) H a H m Then select the constant multipliers so that H a is allpass. otice in the above example that hn ( ) is not real (because the poles and zeros are not in complex-conjugate pairs), and furthermore it is not causal (because if it is stable, the ROC is annular). Consequently, the allpass filter is also not causal. Key idea: The minimum-phase filter has an inverse that has the inverse magnitude frequency response of Hz (). If Hz () has no zeros on the unit circle, then H m will be strictly minimum H m phase, so () z will be stable, causal, and minimum phase. Hence, the factorization in (9) allows us to invert any magnitude frequency response, as long as there are no zeros on the unit circle. H m () z 3. Real-Valued Discrete-Time Fourier Transforms The DTFT ( ) of a sequence hn ( ) is real iff He j ω which is true iff hn ( ) He j ω is conjugate-symmetric, Taing Z transforms on both sides, we see that ( ) H ( e j ω ) (0) hn ( ) h ( n). () He ( j ω ) is real-valued iff Hz () H -. (2) z Since ( ) is real, the phase modulo π is zero, so hn ( ) is sometimes called a zero-phase se- He j ω

5 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 5 quence. ote, however, that this terminology ignores the ambiguity of multiples of π. Equation (2) implies that if there is a zero at z c, then there is also a zero at z /c. Similarly, if there is a pole at z d, then there is also a pole at z /d, as shown in the following diagram: - d d c - c ote that this not the same type of symmetry encountered for allpass sequences, which had polezero pairs at conjugate-reciprocal locations. These symmetry relationships also apply to poles and zeros at zero and infinity. A factor for M 0 in a Z transform introduces M zeros (or poles) at z 0 without matching zeros (or poles) at z, so a zero-phase Z transform cannot have such a term. Correspondingly, except for the trivial case Hz (), a zero-phase sequence is neither causal nor anticausal. Any poles at zero must be matched by an equal number of poles at infinity. Although a real-valued DTFT implies pole-pole and zero-zero pairs at conjugate-reciprocal locations, note that the converse is not true unless the multiplicative constant is carefully selected. Consider the following example, Hz () ( cz )( c z). (3) This has a zero at z c and a zero at z -. It also has a pole at zero (due to the first factor) and c a pole at infinity (due to the second factor). The general form of this example can be written Hz () Az ( )A ---- z z M (4) Evaluating this on the unit circle we see that He jω ( ) Ae ( jω )A ( e jω ) Ae ( jω ) 2 which is not only real, but also non-negative. A related example,, (5) Hz () ( cz )( c z) (6) is real, but not non-negative on the unit circle. A product or ratio of some number of terms of forms

6 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 6 (3) and (6) will be real on the unit circle, but is not necessarily non-negative. 4. Linear-Phase Sequences Suppose where He j ω ( ) is real. Then Fe j ω ( ) e jωτ He j ω ( ) (7) F( ) ωτ modulo π, (8) so fn ( ) is called a linear-phase sequence. Again, the terminology ignores the ambiguity of multiples of π. If τ is an integer, then so that e j ω This might tae a two-sided finite sequence Fz ( ) z Hz () (9) fn ( ) hn ( ). (20) hn ( ) and mae a causal version. 5. Positive Semi-Definite Sequences The sequence hn ( ) with DTFT He ( j ω ) is said to be positive semi-definite (or non-negative definite) if He ( j ω ) is real and He j ω ( ) 0. (2) As with any zero-phase sequence, a pole (or zero) at c implies a pole (or zero) at /c. However, now we also have the stronger result that zeros or poles on the unit circle must be double. Theorem: If the sequence hn ( ) has a real-valued DTFT He ( jω ) and rational Z transform Hz (), then zeros or poles on the unit circle must be double.

7 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 7 Since the proof is rather involved, it is worth giving an intuitive argument first. A single pole or zero on the unit circle causes a phase change of π when the frequency response moves across it. Consequently, if the frequency response is positive on one side of the lone zero or pole, it will be negative on the other side. Showing this formally is a bit more painful. Also note that it follows from this theorem that any positive semi-definite Hz () can be factored as in (4). The theorem, as stated, justifies that the poles and zeros can be separated as in (4). Furthermore, if we assign all the poles and zeros that are inside or on the unit circle to the first factor in (4), we see that we can write the factorization as Hz ( ) H m H m ---- where H m is minimum phase. In the course of proving the above theorem, we will clear out any doubts about the constant multipliers in the above factorization. z (22) Proof: We have already shown that a real DTFT implies that for every pole or zero at c there must be a corresponding pole or zero at the reciprocal conjugate location. This is not sufficient to show that the zeros and poles on the unit circle are double however. To do this, factor the Z transform as follows where form H s Hz ( ) H s H u contains all the poles and zeros that are not on the unit circle, and is written in the (23) H s i ( c i z ) ( c i z) (24) M ( d i z ) ( d i z) i It is easy to verify that this is real-valued and non-negative on the unit circle. For simplicity, we now assume Hz () has no poles on the unit circle (i.e. it is stable). This assumption is not necessary, but it simplifies the proof. In this case, the remaining factor can be written H u Cz L ( a i z ) K i (25) where accounts for all the poles and zeros at zero and infinity not accounted for in (24), z L

8 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 8 a i accounts for all the zeros on the unit circle, and C is some constant. oting that each e jθ i term ( a i z ) accounts for not just a zero at z a i, but also a pole at z 0, we might observe that these poles need to be balanced with poles at infinity for the DTFT to be real. Since a factor z gives us a pole at infinity and a zero at zero, we could conclude that it is necessary that L K/2. Since L must be an integer, we could conclude immediately that K must be even, so the number of zeros on the unit circle must be even. This is true for any zero-phase sequence, regardless of whether it is positive semi-definite. is real-valued iff (2) is satis- The above conclusion can be reinforced by observing that fied, or It is clear from (24) that so we need only ensure that From (25), this requires that He ( j ω ) Hz ( ) H s H u H s ---- Hu ---- H ---- z K H u Cz L ( a i z ) C z L ( a i z ) H u (29) i The polynomial on the left has powers of z ranging from L to L K. The one on the right has powers of z ranging from L to L + K. The only way these two ranges could be the same is if K 2L, which means K is even, as expected. Showing that the zeros must be double zeros is a bit more involved. First observe that it is necessary that H u evaluated on the unit circle be real. Write this as z H s H s ---- z H u H u ---- z K i z z (26) (27) (28) Expanding each term in the product 2L H u ( e jω ) Ce jωl e jθi e jω i. (30)

9 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 9 which leads to where 2L H u ( e jω ) Ce jωl e j θ i i ( ω)/2 j( θ i ω)/2 e H u ( e jω ) Ce js/2 ( ) L sin( ( θ i ω)/2) 2L i e j ( θ i ω)/2 (3) (32) For this to be real, it is necessary and sufficient that. (33), (34) for any real valued constant A. For it to be real and non-negative, it is necessary and sufficient and that the zeros are double, so that we can write (32) as C S C Ae j i where we have numbered the zeros so that θ i θ 2L i. It should be clear that these conditions are sufficient. To show they are necessary, observe that if we have a zero that is not a double zero, then there will be some region ( ε, θ i + ε) that crosses only one zero. The term sin( ( θ i ω)/2) in (32) will change signs in this region, so it cannot be non-negative on both sides of. Q.E.D. 2L θ i Ae js/2 S/2 ( ) L H u ( e jω ) Ce js/2 ( ) L [ sin( ( θ i ω)/2 )] 2 θ i L i θ i (35) (36)

10 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 0 6. Filtering Random Processes For WSS random process xn ( ) and LTI system Hz (), xn ( ) yn ( ) Hz ( ) Define the cross correlations and observe that r xy ( m) Exn [ ( )y ( n m) ] (37) which we can write r xy ( m) h ( )Exn [ ( )x ( n m ) ] h ( )r x ( m+ ) (38) r xy ( m) h ( m)r x ( m). (39) Since the ZT of h ( n) is H ----, the cross power spectrum follows from this z R ( xy z ) H (40) z Rz () The cross correlation is different in the other order, r yx ( m) Eyn [ ( )x ( n m) ] (4) or or r yx ( m) h ( )Exn [ ( )x ( n m) ] h ( )r x ( m ) (42) r yx ( m) h( m)r x ( m). (43) The cross power spectrum in this direction is

11 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL R yx Hz ( )R x. (44) It is easy to show that yn ( ) is WSS, and to compute its power spectrum: r y ( m) E h ( )xn ( ) hi ()xn ( m i) i (45) h ( ) h ()r i x ( m + i ) i h ( )r xy ( m ) (46) so r y ( m) hm ( )h ( m)r x ( m) (47) So the output power spectrum is R y Hz ( )H ---- z Rx. (48) 7. Innovations Process & Spectral Factorization Definition: Assuming xn ( ) is WSS and R x has no poles or zeros on the unit circle. xn ( ) Hz () vn ( ) Whitening filter Causal, monic, and strictly minimum phase. Innovations process 2 white, R v σ v. Define the inverse filter : vn ( ) Gz ( ) xn ( )

12 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 2 where Gz () H. Since vn ( ) is white, samples are uncorrelated, and hence in some sense each one brings new information about the random process xn ( ). When such a representation exists, where Gz () is stable and causal, then xn ( ) is called a linear process. The power spectrum of xn ( ) can be written 2 R x σ vgz ( )G ---- z. (49) This is called the canonical spectral factorization of R x. Knowing the whitening filter and the power of its output implies nowing the power spectrum, because it implies nowing Gz () H, and R x e jω 2 jω ( ) σ v Ge ( ) 2. (50) The standard normalization results from constraining Gz () and Hz () to be monic, i.e. Since these are causal, This defines the value g( 0) and h( 0). (5) G( ) and H( ). (52) unambiguously. For rational polynomial forms, The ey result: Given any rational power spectrum without poles or zeros on the unit circle, there exists a canonical spectral factorization, and hence a monic, minimum-phase whitening filter, and a monic minimum-phase synthesis filter. This is a form of Wold s theorem. The proof is by construction. outside for G ( /z ). σ v 2 Gz () Hz () ( β i z ) ( α i z ) (53) gets the poles and zeros inside the unit circle, leaving the ones

13 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 3 E.g.: Gz ( ) G ---- z S x is positive semi-definite, so zeros on the unit circle are double, and create no difficulties. 8. Power of the Innovations Process Fact: the power of the innovations process equals the geometric mean of the power spectrum: σ v 2 π exp lnr 2π [ x ( ω) ]dω π. (54) This can be recognized as a geometric mean by considering the limiting case of x n n where the sum becomes an integral. From spectral factorization we can write exp - lnx, (55) n n π ln R. (56) 2π [ x ( ω ) ]dω 2 ln( σ v ) ln Ge ( j ω ) 2 dω 2π π π π We need to show that the second term is zero, given that Gz ( ) is monic and minimum phase. Recall that the complex logarithm of a complex number a is

14 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 4 ln( a) lna + j( a+ 2π) (57) for any integer. The principal value uses 0 easy to understand from the form a ae j a ( + 2π). This expression for the complex logarithm is. (58) We do not always need or want the principal value. It is easy to see that we can always pic appropriate values of each of the logarithms (not necessarily the principal values) such that ln( ab) lna ( ) + ln( b). (59) Hence, whenever we write such a relation, we insist that compatible values of the multivalued logarithms be used. Consider ln G ( z )G ---- z ln [ Gz ( ) ] + ln G ---- z. (60) ote that Gz () has all poles and zeros inside or on the unit circle. Hence, it is analytic and nonzero outside the unit circle. This maes ln[ Gz ( )] analytic outside the unit circle, which implies that there exists a causal sequence c ( ) such that ln[ Gz ( )] c ( )z. (6) 0 This sequence is called the complex cepstrum of gn ( ). Although c ( ) is not necessarily stable (it is unstable if Gz () has zeros on the unit circle), it is nonetheless bounded (proven below). Hence, c( 0) ln[ G( ) ]. (62) Since Gz () is monic and causal, G( ) so c( 0) 0. Furthermore, c( 0) is the inverse DTFT of ln[ Ge ( j ω )] evaluated at n 0, so c( 0) π ln Ge ( j ω ). (63) 2π [ ]dω 0 π ote that the imaginary part of this integral is always zero for a real random process xn ( ) as long as we choose values of the multi-valued complex logarithm that are conjugate symmetric about ω 0. We require such a choice.

15 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 5 Use a similar argument for the integral of ln G ----, observing that is anticausal and z G ---- z monic to get Combining these two zero-valued integrals, π ln G ( e j ω ). (64) 2π [ ]dω 0 π Hence, π ln Ge ( j ω ) 2 dω. (65) 2π 0 π π ln (66) 2π [ R x ( ω) ]dω 2 ln( σ v ) π and the desired result follows. The only loose end is to prove that the complex cepstrum cn ( ) is bounded even when it is not stable (i.e. when Gz () has zeros on the unit circle). We will show this only for rational polynomial power spectra, for which Gz () can be written as Gz () i ( α i z ) , (67) M ( β i z ) i where α j and β j <. For this form, ln( Gz ( )) ln ( α i z ) ln( β i z ) i where again we must use compatible values for the complex logarithm. M i Consider each term ln( α i z ). It can be written as a power series expansion (68)

16 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 6 ln( α i z ) n ( α i z ) n n, (69) valid for α i z <, or z > α i. From this expression we can recognize the inverse Z transform, ZT [ ln ( α z )] i 0; n 0 n α i ; n > 0 n (70) ote that although this is not absolutely summable if bounded for all n. Furthermore, α i (and hence is not stable), it is is bounded for all n. A similar argument can be applied to show that the anticausal sequence is also bounded. ZT [ lngz ( ())] cn ( ) M i α i n ZT ln G ---- z 0; n 0 β i n ; n > 0 n n i (7). (72)

17 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 7 9. Wiener Filters sn ( ) signal noise xn ( ) LTI fn ( ) yn ( ) filter wn ( ) our design dn ( ) en ( ) Objective is to minimize the power in en ( ) (MSE criterion). Example applications:. dn ( ) sn ( ) (try to minimize the noise) 2. dn ( ) sn ( + m) (try to predict sn ( )) 3. dn ( ) xn ( + m) (try to predict xn ( )) Assumption: xn ( ) and dn ( ) are jointly WSS ( Exn [ ( )d ( n m) ] is independent of m ). Constraint: fn ( ) is usually, but not always, constrained to be causal. 0. Wiener-Hopf Equations Minimize the mean-squared error (MSE): Expanding this, ε 2 E[ d( n) yn ( ) ]. (73) ε Edn [ ( )d ( n) ] (74)

18 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 8 Hence f ( )Edn [ ( )x ( n ) ] + m f ( )Ed [ ( n)xn ( ) ] fm ( )f ( )Exn [ ( m)x ( n ) ] ε r d ( 0) 2Re f ( )r dx ( ) + fm ( )f ( )r x ( m) m (75) (76) (77) This is a real-valued quadratic function of the possibly complex variables fi () for all i. This suggests that by taing the partial derivative of ε with respect to each fi (), and setting this to zero, we can find that values of fi () that minimize ε. However, this derivative in general does not exist. In particular, df does not exist anywhere. However, note that ε is a real-valued quadratic function df of the real and imaginary parts f R () i and f I () i of fi () for all i. Hence, we can tae the partial derivatives with respect to each of these real-valued variables and set those to zero. The values of fi () that mae the partial derivatives zero yield a minimum ε if the function is unimodal. We assume for now that it is. Instead of setting the partial derivatives to zero, we can equivalently set the following complex gradients to zero, fi () ε - j 2 f R () i f I () i ε 0 (78) and. ote that the complex gradient has other definitions, such as ε. This would serve equally f + j f R f I ε well for our optimization problem and is used in [S. M. Kay, Modern Spectral Estimation, Prentice-Hall, 988] and [S. Hayin, Adaptive Filter Theory, 2nd Ed., Prentice-Hall, 99]. Our definition is used in [C. W. Therrien, Discrete Random Signals and Statistical Signal Processing, Prentice-Hall, 992] and [D. H. Brandwood, A Complex Gradient Operator and its Application in Adaptive Array Theory, IEE Proceedings, 30(), pp. -6, February 983]. The ey advantage of our definition is that if the function being differentiated is an analytic ε function of f, then ε. f f

19 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 9 f () i ε - + j 2 f R () i f I () i ε 0 (79) Equivalence follows from the observation that f () i ε + fi () ε ε f R () i (80) and With our definition of the complex gradient, it is easy to verify that and Using these we get It is easy to show that ε fi () ε j ε. (8) f I () i f 0, f, and ff f (82) f, f 0, and ff f. (83) (using the conjugate symmetry of r x ( m) ), so it is sufficient to set only one of the two gradients to zero (either one). Setting f f f () i f () i f f f f ε r dx () i + f ( )r x ( i ). (84) ε 0 f () i fi () ε ( ε) f () i we get the Wiener-Hopf Equations: (85) f ( )r x ( i ) r dx () i, for all i (86) A solution to these equations will minimize ε. We can easily modify these equations to constrain the filter fn ( ) to be causal by just modifying the limits on the summation,

20 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 20 0 f ( )r x ( i ) r dx () i, for i 0. (87). Some Easy Examples:. Suppose dn ( ) xn ( ). Then r dx () i r x () i. The Wiener-Hopf equations are satisfied by fn ( ) δ( n). (88) Interpretation: xn ( ) δ( n) dn ( ) en ( ) 2. Suppose dn ( ) sn ( ) xn ( ) wn ( ), where wn ( ) is white and independent of xn ( ) (a very strange situation. Again r dx () i r x () i, so the solution is the same. Interpretation: sn ( ) signal noise xn ( ) LTI fn ( ) yn ( ) filter wn ( ) dn ( ) en ( ) Because xn ( ) is independent of the noise wn ( ), there is nothing the filter can do to remove it. So the solution is the same as before.but a more realistic scenario would be for wn ( ) to be independent of sn ( ). 3. Suppose xn ( ) is white. I.e. r x ( ) σ 2 δ( ). Then the Wiener-Hopf equations are satisfied by

21 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 2 fn ( ) r dx ( n) un ( ), (89) σ 2 where un ( ) is the unit step function (so this filter is causal). Knowing the joint statistics of xn ( ) and dn ( ), we can find the MMSE filter that estimates dn ( ) from xn ( ). The solution is easy when xn ( ) is white. For other situations, we can use a whitening filter to construct a general solution. 2. on-causal Wiener Filter Solution Equation (86), which has no causality constraint on the Wiener filter, can be written Taing Z transforms and solving for fi ()r x () i r dx () i. (90) Fz ( ) Fz ( ) we get R dx () z (9) R x This is sometimes called the unrealizable Wiener filter, but we prefer to call it the non-causal Wiener filter. 3. Causal Wiener Filtering, Rational Power Spectra The causality constraint in (87) complicates things considerably. Conceptually brea the filter into two parts: sn ( ) xn ( ) vn ( ) yn ( ) hn ( ) f ( n) en ( ) wn ( ) fn ( ) dn ( )

22 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 22 where hn ( ) is the whitening filter for xn ( ), vn ( ) is the innovations process, and f ( n) is the Wiener filter designed for the white input. I.e. otation: f ( n) r dv ( n) un ( ) (I.e., the Z transform of the nonnegative side only). With this σ v 2 (92) [ R dv ] + r dv ( )z. (93) F 0 - [ R dv ] +. (94) σ v 2 A more convenient form would express F as a function of R dx, which is assumed to be nown. 4. Derivation of the General Solution ote that since vn ( ) h ( )xn ( ) 0 r dv ( m) Edi [ ()v ( i m) ] (95) (96) 0 h ( )Edi [ ()x ( i m ) ] (97) This relation can be written 0 h ( )r dx ( m+ ). (98)

23 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 23 r dv Taing Z transforms on both sides, ( m) h ( m)r dx ( m). (99) or R dv - z H ( z )R dx - z (00) R dv H z Rdx (0) Hence, the overall Wiener filter solution is Fz ( ) Hz ( )F - Hz ()H ---- z Rdx σ v 2 + (02) 5. A More Involved Example 2 Assume in the following that un ( ) is white noise, or R u σ u : un ( ) az sn ( ) xn ( ) yn ( ) fn ( ) en ( ) wn ( ) dn ( ) 2 Assume further that wn ( ) is white noise ( R w σ w ), uncorrelated with sn ( ) dn ( ). In words, we are given noisy observations of a first-order AR process, and need to design a Wiener filter to remove the noise as much as possible.

24 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL Linear Equalization Scenario: dn ( ) xn ( ) yn ( ) en ( ) Cz () Fz ( ) The channel Cz ( ) is assumed to be causal and stable. The data dn ( ) is assumed to be white with power spectrum. Our tas is to design the Wiener filter Fz ( ) to minimize the power in σ d 2 en ( ). The general solution is Fz ( ) (03) where is the power of the innovations process for xn ( ), and Hz () is the whitening filter. To σ v 2 get a more specific result, observe that we can factor component, where by definition we scale these so that - Hz ( ) H ---- z Rdx The whitening filter (which by definition is monic) is σ v 2 Cz ( ) into an allpass and a minimum phase Cz ( ) C m C a, (04) C a ( e jω ). (05) Hz () KC m, (06) where K is the constant needed to mae it monic. The innovations process is generated by the system: + dn ( ) Cz ( ) xn ( ) vn ( ) Hz ( ) ote that since Cz ( )Hz ( ) C a ()K z, where C a is allpass, then

25 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 25 Moreover R ( dx z ) C (08) z Sd σ dca ---- Cm ---- Putting this all together, plugging (06), (07), and (08) into (03) we get. (09) If Cz ( ) happens to be minimum phase, i.e. Cz ( ) C m and C a, then intuitively we expect Fz ( ) C () z, which is exactly what we get. If it is not minimum phase, however, then we have the term C a ote that the inverse Z transform of C a ---- is c a ( n). But since C a causal, the non-negative time part of this consists of exactly one sample, c a ( 0)δ( n). Hence and we can write the final solution as As a specific example, suppose (0) Fz ( ) c a ( 0)Cm. () (2) where p <, so the system is stable, but c >, so the system is not minimum phase. In this case, and z + To verify this, observe that the product equals Cz ( ), that C a evaluates to unity on the unit cirσ v σ d K. (07) Fz ( ) C m ()C z a ---- C a ---- z Cz ( ) C m C a + (3) z c a ( 0) cz pz c + z pz z + z cz (4) c + z z

26 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 26 cle, and that C m is minimum phase. To find c a ( 0) we can use the final value theorem, evalu- ating C a ---- at z 0 (because it is anti-causal). This is z Plugging this into () we get C a ---- Fz ( ) z z 0 pz c + z c z c + z. (5) pz. (6) + ( c ) z We see that instead of inverting the zero outside the unit circle, we invert its conjugate reciprocal. Furthermore, the closer c gets to the unit circle, the closer the solution gets to the solution when Cz ( ) is minimum phase. Moreover, the further c gets from the unit circle, the closer the Wiener filter gets to zero. This latter result is intuitive because the filter is getting further from minimum phase, so the inverse of the minimum phase part is less useful. c 2 z 0 - c 7. ormal Equations Let f be a vector of dimension M, representing the taps of an FIR Wiener filter (i.e. the first M values of the causal impulse response). For this filter to be optimal, the taps must satisfy the normal equations, R x f. (7) Assuming that these equations result from minimizing a unimodal quadratic function, it follows that there must be at least one solution for f. This conclusion can be reinforced algebraically by r dx proving that is always in the range space of. We write this r dx R( R x ), (8) where R( R x ) is simply the set of vectors that can be written as R x p for some p. r dx R x Consider a vector q in the nullspace of R x, meaning that R x q 0, where 0 is the zero vector. This is written

27 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 27 For this vector, q η( R x ). (9) q H R x q 0. (20) Letting x( n) [ xn ( ), xn ( ),...,x( n M+ ) ] T, we can write Hence R x E[ x( n)x H ( n) ]. (2) or q H E[ x( n)x H ( n) ]q 0 (22) E[ q H x( n)x H ( n)q] 0. (23) oticing that this is the product of two complex scalars that are complex conjugates of one another, we can write it as E[ q H x( n) 2 ] 0. (24) From this we infer that q H x( n) 0 with probability one. In words, with probability one, an outcome of the random vector x( n) will be orthogonal to any vector q in the nullspace of R x. ow, note that so that r dx q H r dx Ed [ ( n)x( n) ] (25) Ed [ ( n)q H x( n) ]. (26) But since q H x( n) 0 with probability one, this inner product is zero, so r dx is orthogonal to any vector q in the nullspace of. The following two facts from linear algebra let us infer directly R x H from this that is in the range space of R x. These facts use the property that R x R x : r dx R( R x ) ηr ( x ) and R( R x ) η( R x ). (27) C M

28 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL Geometric Series n m 0 a m a n + for any n > 0 (28) a To prove, just multiply both sides by a. ote that, taing the limit as n,. Dirac Delta Functions The Dirac delta function and δ() t m 0 a m , for any a <. (29) a is defined by the following two relations: δ() t 0 for all t 0 (30) From this definition, we can get the following identity δ()dt t. (3) aδ( at) δ() t (32) valid for any real a. To prove this, just observe that and aδ( at) 0 for all t 0 (33) aδ ( at )dt. (34) The latter is obtained by a change of variables, where t gets replaced by at and hence dt get replaced by adt.

29 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL Woring with Impulse Trains The following identity relates an infinite sum of complex exponentials with an infinite Dirac delta train: e jωn n 2πδ( ω 2π). (35) It can be verified by observing that the left hand side is the DTFT of the signal xn ( ), from the definition of the DTFT. So applying the inverse DTFT formula to the right hand side, we should obtain xn ( ). Doing this, xn ( ) π π δ( ω 2π)e jωn dω. (36) 2π π ote that the limits of the integral enclose exactly one of the impulses in the train (the one at ω 0 ). So applying the sifting rule, we get xn ( ). Another form of the same identity relates a delta train in the time domain with a sum of exponentials in the frequency domain: δ( t T) - e T m j 2π mt T (37) We can verify this by changing variables in the previous identity. But for extra reinforcement, let us verify it independently by letting xt () δ( t T) (38) and viewing the sum of exponentials as a Fourier series expansion for the periodic function xt (), which has period T. Find the Fourier series coefficients using the formula Xm ( ) T/2 T/2 j 2π mt T - xt ()e dt T - δ( t T) T e dt. (39) Again, the integral encloses just one of the delta functions, so we apply the sifting rule. A more T/2 T/2 j 2π mt T

30 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 30 formal way to simplify the above expression is to exchange the integral and summation and change variables, letting τ t T, getting Xm ( ) T + T/2 j 2π m T ( τ + T ) - δτ ( )e dτ T T T/2 We can recognize this as a sum of integrals with adjoining limits and simplify to Xm ( ) j 2π mt T - e T (40) -. (4) T Since the Fourier Series coefficients are constant, the Fourier Series representation of xt () is an infinite sum of exponentials with equal weight. From this representation, it is easy to see that the Fourier transform of xt (), which can itself be written as a sum of exponentials, can also be expressed as a sum of Dirac delta functions, FT δ( t T) e jωmt m 2π δ ω 2π T T m m (42) The first of these expressions comes from the Fourier integral and the sifting rule of Dirac delta functions. The second follows from the first identity (after some manipulation), or can be derived from nowledge that j 2π T mt FT e 2πδ ω 2π m T (easily verified using the inverse Fourier integral) and the fact that such terms. Substituting in place of T we get the related identity xt () (43) is a linear combination of m e jωm 2π δ ω 2π m m (44) The first of these two forms, is easily recognized as the DTFT of Xe jω ( ) e j m ωm, (45)

31 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 3 xn ( ) δ( n m) m (46) where δ( n) is now the Kronecer delta function. (To verify this, just note that the DTFT of each component δ( n m) is e jωm, from the DTFT definition). If we apply the inverse DTFT formula to the second of the two forms for tity for Kronecer delta trains Xe jω ( ), after some non-trivial labor, we get the new iden- m δ( n m) - e 0 j 2π n (47) ote that unlie the corresponding identity for Dirac delta trains, the sum of exponentials is finite. To verify this more easily than applying the inverse DTFT formula, note that the finite sum of exponentials is exactly the inverse DFT of X ( ), as is easily seen by comparing the above expression to the inverse DFT formula. Hence, we can just verify that DFT[ xn ( )], where we use the DFT of length. (Equivalently, we could view the above sum of exponentials as a discrete Fourier series (DFS) and just verify the DFS coefficients.) X ( ) DFT[ xn ( )] δ( n m) n 0 m e j 2π n (48) Exchanging summations, and changing variables using r n m we get m+ X ( ) δ( r)e m r m j 2π r m ( + ) The double summation is actually a sum of sums with adjoining limits, so (49) j 2π r m ( + ) X ( ) δ( r)e e Thus our identity is verified. A side benefit of this approach is that we have verified that r j 2π (50)

32 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL 32 DFT δ ( n m ) m (5) for a DFT of order. We can show similarly that DFT[ ] δ( m ), (52) again for a DFT of order. We can now use identity (47) to find the DTFT of Applying the DTFT definition, xn ( ) δ( n m). (53) m Xe jω ( ) - e 0 j 2π n Exchanging the order of summations and combining the exponentials, Xe jω ( ) jωn e. (54). (55) We can now use identity (35) to represent the inside summation as a delta train, Xe jω ( ) n - e 0 0 m n Exchanging summations again, we can recognize this as a sum of sums with adjoining limits, getting the simpler expression j ω 2π n 2π (56) δ ω 2π m + - DTFT δ( n m) m 2π δ ω 2π - (57) As a picture:

33 EE 225A DIGITAL SIGAL PROCESSIG SUPPLEMETARY MATERIAL π 2π (58) The DTFT of a discrete-time impulse train is an impulse train in frequency, just as the FT of a continuous impulse train is an impulse train in frequency. ω