Lesson 1 Optimal Signal Processing Optimal signalbehandling LTH September 2013 Statistical Digital Signal Processing and Modeling, Hayes, M: John Wiley & Sons, 1996. ISBN 0471594318 Nedelko Grbic Mtrl from Bengt Mandersson Department of Electrical and Information Technology, Lund University Lund University
The sound of Signalbehandling s i g n s noise i harmonic signal How can this be generated as output from a linear filter? Determine the filter and the input signal. LPC model of syntetic sound production pulse train speech output from pulse train white noise H 0 z LPC-model speech output from white noise waveform and spectra In syntetic speech production, the parameters often are updated every 5 milliseconds. 2
Chapter 2. Digital signal processing impulse response, convolution, system function, Fourier, z-transforms page 7-20 Matri description. page 20-52 Hints. page 8-18, 21, 49. Chapter 3. Random processing, such as correlation functions, correlation matrices. Random variables page 58-74 Random Processes page 74-119 Hints. page 77, 79, 80, 85, 95, 99, 100, 101, 106 Chapter 4. Signal models, Deterministic and Stochastic approach. Padé, Prony page 133-154 Shank page 154-160 All-pole Modeling page 160,165 Linear prediction page 165-174 4.5 not included 4.6 page 178-188 4.7 Stochastic Models page 188-200 Hints. page 130, 135, 138, 147, 148,149 195, 195 Chapter 5. Levinson-Durbin recursion. page 215-225, 233-241 page 242 276 not included Hints. Table 5.1 5.4, figure 5.10 Chapter 6. Latttice FIR and IIR filters, only 6.2 and 6.4.1, 6.4.3 page 289-293, 297, 298, 304-307 6.5 page 308-324 Chapter 7. Optimal filters. Linear prediction. Wiener filters. Specially FIR filters. FIR- Wiener filter page 335-345 IIR- Wiener filter page 353-371 Kalman filters page 371-379 Hints. page 337-339, 354, 355, 358-363, 370 Chapter 8. Spectrum estimation. Nonparametric methods page 393-399, 408-425 8.3 8.5 see chap 4, 8.6 page 426-429, 451-472 Hints. page 394, 408 3
Digital Signal Processing application Radar 4
An application from the tet book Noise cancellation chapter 7, page 349 A signal is disturbed by additive noise v 1 n. Try to measure the noise vn from the source and estimate the noise v 1 n added to the signal. Then subtract the noise v 1 n from the received signal. Signal source Noise source vn vn sn sn+v 1 n sn Hz v 1 n Wiener filter Estimate of v 1 n 5
Optimal signal processing in Hay's book Chapter 2: Chapter 3: Brief review of digital signal processing. Brief review of random signals. noise vn white noise wn or impulse δn transmitted h gen n sn H gen z received signal n h receiver n H receiver z yn Estimate H gen z from properties of sn Determine H receiver z The filters H gen z and H receiver z are of type FIR IIR all-pole IIR Chapter 4, 5 and 6: Make a model H gen z from the properties of sn. Chapter 7: Chapter 8: Determine H receiver z. Estimation of spectra. 6
Chapter 2 Digital Signal Processing Difference equation p y n a k y n k b k n k k 1 q k 0 MATLAB: A=[1 0.5 0.5]; B=[1 1]; y=filterb,a,; Convolution k y n h k n k impulse: n [0 0 0 1 0 0 0] unit step: un [0 0 0 1 111...] System function H z Frequency function B z A z j j B e H e j A e 7
FIR, IIR filters FIR: length Circuit with impulse response with finite Eample y n n n 1, h n n n 1 IIR: Circuit with impulse response with infinite length Eample n y n 0.5 y n 1 n, h n 0.5 u n All-pole IIR-filters IIR-filters with poles only all zeroes in origin, Bz=constant Eample H 1 1 0.5 z z 1 8
Solvning the convolution sum. y n h k n k k h n k k k y n h0 n h1 n 1 h2 n 2 Eample n [1 2 3 4], h n [4 2 2] Method A: Vector notation n n 1 y n n. n N 1 T T h0 h1... h N 1. h Method B: Graphical solution Write k : 1 2 3 4 h0 k : 2 2 4 y0 41 4 h1 k : 2 2 4 y1 21 42 10 Gives the output yn [4 10 18 26 14 8] MATLAB: =[1 2 3 4]; h=[4 2 2]; y=conv,h 9
Method C: Convolution matri Use matri notations n [1 2 3 4], h n [4 2 2] 0 0 0 1 0 0 h 0 2 1 0 h 3 2 1 1 h 2 0 3 2 0 0 3 y 0 y 1 y 2 y 3 y 4 y5 1 0 0 2 1 0 4 3 2 1 2 4 3 2 2 0 4 3 0 0 4 4 10 18 26 14 8 X h y In Matlab: =[1 2 3 4] ; X=convmt,3 h=[4 2 2]', y=x*h In signal processing, all vectors are column vectors 10
Properties of matrices The square matri A n n is: symmetrical if A A T T Hermitian if A A A H invertable if AA 1 I Toeplitz if all diagonals are identical 345 A 234 1 2 3 Hermitian symmetrical Toeplitz if 3 2 1 A 232 1 2 3 A Toep[3, 2,1] orthogonal if T A A I 11
Linear equation page 31-34 A is a A [ n m] matri b gives 1 A b if n m A invertable, 1 H H A A A b if n m overdetermined, more equations than variables. Described more in chapter 4 A H A A H 1 b if n m underdetermined, less equations than variables Eigenvalue: Av v, A I 0 eigenvalues, v eigenvectors A V V 1 with eigenvectors in the columns of V, eigenvalues in the diagonal of 12
Optimisation minimizing: page 49 If z real: f z z 2 d d 2 d 2 f z z 2 z; z 0 dz dz dz gives z 0 as min imum; If z is comple: 2 f z z z z z is the conjugate of z z Derivate with respect to treating the other as a constant. or z separately while d dz d dz d z z z z dz d z z z z dz 2 2 Setting this derivatives equal to zero gives the same minimum page 49. This is used sometimes in the tetbook. 13
Eample on circuits A n Xz z -1 yn Yz 0.5 y n 0.5 y n 1 n 1 Y z 0.5 z 1 Y z z 1 X z B n Xz z -1 z -1 yn Yz 0.5 0.5 0.5 C Lattice filters n Xz Г 1 z -1 Г 1 z -1 Г 2 Г 2 yn Yz FIR-lattice filter n, Xz yn, Yz -Г 2 -Г 1 Г 2 z -1 Г 1 z -1 IIR-lattice filter 14
Correlation functions deterministic Autocorrelation function r l n n l r l n Cross-correlation function r l y n n l y n r l l l ry l y l l Relation between input and output r l h l r l y r l r l r l y h 15
Eample on correlation, echo 1 2 y= 1 + 2 r 1 r y r 1y 16
Eample of correlation, delay in mobile phones GSM Input signal to the GSM phone Output signal after GSM Crosscorrelation In Matlab: ry=corrinput,output 17
Chapter 3 Discrete-Time Random Processes Random variables 3.2 page 58-74 Probability density function f X Probability distribution function: F X Epected value mean: Mean-square value: m E{ } f X d 2 2 E{ } f X d Variance: Var E m m f d 2 2 2 [ ] {[ ] } [ ] X General: y g ; E{ y} E{ g } g f X d Relation: Var[ ] E{[ m] } E{ X } m 2 2 2 Correlation. Dependency between random variables and y Correlation: r y E{ y} 18
Covariance: c y E {[ m ][ y m y]} 19
Stochastic processes 3.3 page 74 Wide-sense stationary processes, WSS Eample A: Sinusoids with random phase n Asin 0 n, is a random variable and n is a random process. Eample B: Noise white noise, colored noise. Eample C: Speech signals. The autocorrelation sequence and the cross-correlation sequence and their Fourier transforms are important in this course. Autocorrelation sequence: Cross-correlation sequence. { r } m E k k m { r } y m E k y k m Estimation of the autocorrelation sequence ergodic processes 1 r { } m E k k m k k m N sum over N values 20
Interpreting of autocorrelation sequence: Signal Autocorrelation sequence Sinusoid: White noise. Colored noise Speech signal: Vowel. 21
22 Optimal Signal Processing Properties of autocorrelation sequence page 83 Wide-sense stationary processes, WSS Definition: k r k n n E n k n E k n n E k n n E k r Symmetry: r k r k Mean-square value: 2 0 [ ] 0 r E n positive Maimum value: 0 r r k Non-stationary processes For signals that are not wide-sense stationary processes, not WSS, we have to use the definitions see chapter 4 } {, } {, * * l k y E l k r l k E l k r y
Correlation matri WSS [ 0 1... N 1] T R H E[ ] r0 r 1 r 2 r p r 1 r 0 r 1 r p 1 r 2 r 1 r 0 r p 2 r p r p 1 r p 2 r 0 Properties of the correlation matri Hermitian Toeplitz Toeplitz if real-valued process Eigenvalues are real and non-negative Estimate of the correlation function N 1 1 rˆ k n n k N n0 Estimate of the cross-correlation function rˆ y k 1 N N 1 n0 n y n k 23
Power spectrum of random process 3.3.8 page 94: Wide-sense stationary processes, WSS n is a wide sense stationary random process WSS, n real-valued, hn real with autocorrelation r k The Fourier transform and the z-transform are given by: The Fourier transform of r k The Z-transform of r : j P e r k e k : P z r k z k j k Properties Symmetry real processes j j : P e P e Positive: Total power: j P e 0 1 j r0 P e d 2 24
Filtering of random processes, 3.4 page 99, 100, 101: n r k hn He j yn r y k Input-output relation y n n h n k h n k k Autocorrelation function for the output r k E{ y n y n k} h l r m l k h m y l m Cross correlation functions r k E{ y n n k} h l r k l y l r y k E{ n y n k} l h l r k l 25
Using convolution and power spectra Define r h k h l h l k h k h k Correlation functions r k r k h k h k r k r k y h r k r k h k r y y k r k h k Spectra j j j P e P e H e y j j j P e P e H e P y y e j P e j H e j 2 1 Py z P z H z H z 26
Spectral factorization 3.5 page 104 n is a WSS process with autocorrelation r k. We assume that the process are generated from white noise vn filtered in a filter with system function Qz, Then, vn is called the innovation process of the process n. white noise vn Qz our process n 1/Qz white noise vn r v k 2 r k k v 2 P z v 0 0 r k 2 P z 0 Q z Q 1/ z Can we find the filter Qz from n and r k? Is Qz stable and causal? Is 1/Qz stable and causal? 27