ESE 531: Digital Signal Processing Lec 25: April 24, 2018 Review Course Content! Introduction! Discrete Time Signals & Systems! Discrete Time Fourier Transform! Z-Transform! Inverse Z-Transform! Sampling of Continuous Time Signals! Frequency Domain of Discrete Time Series! Downsampling/Upsampling! Data Converters, Sigma Delta odulation! Oversampling, oise Shaping! Frequency Response of LTI Systems! Basic Structures for IIR and FIR Systems! Design of IIR and FIR Filters! Filter Banks! Adaptive Filters! Computation of the Discrete Fourier Transform! Fast Fourier Transform! Spectral Analysis! Wavelet Transform! Compressive Sampling 2 Digital Signal Processing! Represent signals by a sequence of numbers " Sampling and quantization (or analog-to-digital conversion)! Perform processing on these numbers with a digital processor " Digital signal processing! Reconstruct analog signal from processed numbers " Reconstruction or digital-to-analog conversion digital digital signal signal analog signal A/D DSP D/A Analog input # analog output Eg. Digital recording music Analog input # digital output Eg. Touch tone phone dialing, speech to text Digital input # analog output Eg. Text to speech Digital input # digital output Eg. Compression of a file on computer analog signal Discrete Time Signals and Systems 4 Signals are Functions Discrete Time Systems 5 6 1
System Properties LTI Systems! Causality " y[n] only depends on x[m] for m<=n! Linearity " Scaled sum of arbitrary inputs results in output that is a scaled sum of corresponding outputs " Ax 1 [n]+bx 2 [n] # Ay 1 [n]+by 2 [n]! emoryless " y[n] depends only on x[n]! Time Invariance " Shifted input results in shifted output " x[n-q] # y[n-q]! BIBO Stability " A bounded input results in a bounded output (ie. max signal value exists for output if max ) 7! LTI system can be completely characterized by its impulse response! Then the output for an arbitrary input is a sum of weighted, delay impulse responses y[n] = x[n] h[n] 8 DTFT Definition Discrete Time Fourier Transform X (e jω ) = x[n] = 1 2π k= π π x[k] e jωk X (e jω )e jωn dω Alternate X ( f ) = x[n] = 0.5 0.5 k= x[k] e j2π fk X ( f )e j2π fn df 10 Example: Window DTFT Example: Window DTFT Also, Σx[n] ( ) sin( ω 2) sin ( +1 2)ω W (e jω ) = W (e jω ) = k= w[k] = e jωk k= e jωk Plot for =2 =1 why? 11 12 2
LTI System Frequency Response z-transform! Fourier Transform of impulse response x[n]=e jωn LTI System y[n]=h(e jωn )e jωn H(e jω ) = k= h[k] e jωk! The z-transform generalizes the Discrete-Time Fourier Transform (DTFT) for analyzing infinitelength signals and systems! Very useful for designing and analyzing signal processing systems! Properties are very similar to the DTFT with a few caveats 13 14 Complex Exponentials as Eigenfunctions Region of Convergence (ROC) z n H(z)z n H(z)z n 15 16 Inverse z-transform Partial Fraction Expansion! Ways to avoid it: " Inspection (known transforms) " Properties of the z-transform " Partial fraction expansion " Power series expansion! Let X (z) = b k z k a k z k! zeros and poles at nonzero locations = z z b k z k a k z k 17 18 3
Partial Fraction Expansion Example: 2 nd- Order z-transform! If < and the poles are 1 st order! 2 nd -order = two poles Right sided! where X (z) = b 0 k=1 a 0 k=1 (1 c k z 1 ) A = k (1 d k z 1 ) k=11 d k z 1 A k = (1 d k z 1 )X (z) z=d k 1 2 X (z) = 1 + 1 4 z 1 1, ROC = z : 1 2 < z 1 2 z 1 x[n] = 1 n 4 u[n]+ 2 1 n 2 u[n] 19 20 Partial Fraction Expansion Example: Partial Fractions! If and the poles are 1 st order! ==2 and poles are first order A k X (z) = B r z r + r=0 k=11 d k z 1! Where B k is found by long division A k = (1 d k z 1 )X (z) z=d k A 1 A 2 X (z) = B 0 + 1 1 + 1 z, 1 2 z 1 1+5z 1 X (z) = 2 + (1 1 2 z 1 )(1 z 1 ) ROC = { z : 1< z } 1 2 z 2 3 2 2 z 1 +1 z 2 + 2z 1 +1 z 2 3z 1 + 2 5z 1 1 21 22 Example: Partial Fractions Power Series Expansion! ==2 and poles are first order! Expansion of the z-transform definition 9 8 X (z) = 2 1 1 + 1 z, 1 2 z 1 ROC = { z : 1< z } X (z) = n= x[n]z n =!+ x[ 2]z 2 + x[ 1]z + x[0]+ x[1]z 1 + x[2]z 2 +! x[n] = 2δ[n] 9 1 n 2 u[n]+8u[n] 23 24 4
Example: Finite-Length Sequence Difference Equation to z-transform! Poles and zeros? X (z) = n= X (z) = z 2 1 1 2 z 1 (1+ z 1 )(1 z 1 ) = z 2 1 2 z 1+ 1 2 z 1 1, n = 2 1 2, n = 1 x[n] = 1, n = 0 x[n]z n 1 2, n =1 0, else =!+ x[ 2]z 2 + x[ 1]z + x[0]+ x[1]z 1 + x[2]z 2 +! a k y[n k] = b m x[n m] m=0 a y[n] = k y[n k] + k=1 a 0 b k a 0 x[n m]! Difference equations of this form behave as causal LTI systems " when the input is zero prior to n=0 " Initial rest equations are imposed prior to the time when input becomes nonzero " i.e y[-]=y[-+1]= =y[-1]=0 25 26 Difference Equation to z-transform a y[n] = k y[n k] + a k=1 0 b k a 0 x[n m] Example: 1 st -Order System y[n] = ay[n 1]+ x[n] H(z) = ( b k ) z k m=0 a k ( ) z k a Y (z) = k a z k Y (z) + k=1 0 a k b a z k Y (z) = k 0 a z k X (z) Y (z) = 0 H(z) = b k a 0 ( b k ) z k m=0 a k ( ) z k z k X (z) ( b k ) z k m=0 a k ( ) z k X (z) 27 1 H(z) = 1 az 1 h[n] = a n u[n] 28 DSP System Sampling and Reconstruction 29 30 5
Ideal Sampling odel Ideal Sampling odel! Ideal continuous-to-discrete time (C/D) converter " T is the sampling period " f s =1/T is the sampling frequency " Ω s =2π/T 31 32 Frequency Domain Analysis Frequency Domain Analysis X s (e jω ) X s (e jω ) ω = ΩT ω = ΩT Ω s 2 T = π 33 Ω s 2 T = π 34 Aliasing Example Reconstruction of Bandlimited Signals! yquist Sampling Theorem: Suppose x c (t) is bandlimited. I.e.! If Ω s 2Ω, then x c (t) can be uniquely determined from its samples x[n]=x c (nt)! Bandlimitedness is the key to uniqueness ulitiple signals go through the samples, but only one is bandlimited within our sampling band 35 36 6
Reconstruction in Frequency Domain Reconstruction in Time Domain * = The sum of sincs gives x r (t) # unique signal that is bandlimited by sampling bandwidth 37 38 Anti-Aliasing Filter DT and CT processing X C ( jω) 1 1/T X S ( jω) Ω S /2 -Ω Ω Ω X C ( jω)x LP ( jω) 1 1/T X S ( jω) Ω S /2 -Ω Ω Ω Ω S /2 39 Discrete-Time Processing of Continuous Time Impulse Invariance x[n] y[n]! Want to implement continuous-time system in discrete-time X (e jω ) = 1 T T k= X c ω j T 2πk T T y r (t) = y[n] n= sin[π (t nt ) / T ] π (t nt ) / T! If x c (t) is bandlimited by Ω s /T=π/T, then, Y r ( jω) X c ( jω) = H ( jω) = H(e jω ) Ω < Ω s / T ω=ωt eff 0 else 41 42 7
Impulse Invariance! With H c (jω) bandlimited, choose H(e jω ) = H c ( jω / T ), ω < π Continuous-Time Processing of Discrete-Time! Useful to interpret DT systems with no simple interpretation in discrete time! With the further requirement that T be chosen such that H c ( jω) = 0, Ω π / T h[n] = Th c (nt ) 43 x c (t) = x[n] n= sin[π (t nt ) / T ] π (t nt ) / T y c (t) = y[n] n= sin[π (t nt ) / T ] π (t nt ) / T 44 Example: on-integer Delay Example: on-integer Delay! What is the time domain operation when Δ is noninteger? I.e Δ=1/2! y delay system has an impulse response of a sinc with a continuous time delay 45 46 Downsampling Rate Re-Sampling! Definition: Reducing the sampling rate by an integer number 47 48 8
Example Example 2π 4π 2π 4π 6π 49 50 Example Upsampling! Definition: Increasing the sampling rate by an integer number x[n] = x c (nt ) x i [n] = x c (nt ') 51 52 Upsampling Example x i [n] 53 54 9
on-integer Sampling Interchanging Operations! T =T/L " Upsample by L, then downsample by interpolator decimator Upsampling -expanding in time -compressing in frequency Downsampling -compressing in time -expanding in frequency 55 56 Interchanging Operations - Summary Polyphase Decomposition Filter and expander Expander and expanded filter* Compressor and filter Expanded filter* and compressor *Expanded filter = expanded impulse response, compressed freq response 57 58 Polyphase Implementation of Decimator Polyphase Implementation of Interpolation interpolator decimator interpolator decimator E 0 (z) E 0 (z) E 0 (z) 59 60 10
ulti-rate Filter Banks Perfect Reconstruction non-ideal Filters! Use filter banks to operate on a signal differently in different frequency bands " To save computation, reduce the rate after filtering! h 0 [n] is low-pass, h 1 [n] is high-pass " Often h 1 [n]=e jπn h 0 [n] $ shift freq resp by π 61 62 Quadrature irror Filters Frequency Response of Systems Quadrature mirror filters 63 64 Frequency Response of LTI System Y ( e jω ) = H ( e jω ) X ( e jω )! We can define a magnitude response ( ) = H ( e jω ) X ( e jω ) Y e jω Group Delay! General phase response at a given frequency can be characterized with group delay, which is related to phase! And a phase response Y ( e jω jω jω ) = H ( e ) + X ( e ) ω 1 ω 2 ω 65 - slope 66 11
LTI System General All-Pass Filter! d k =real pole, e k =complex poles paired w/ conjugate, e k * Stable and causal if all poles inside unit circle! Transfer function is not unique without ROC " If diff. eq represents LTI and causal system, ROC is region outside all singularities " If diff. eq represents LTI and stable system, ROC includes unit circle in z-plane 67 68 inimum-phase Systems in-phase Decomposition Purpose! Definition: A stable and causal system H(z) (i.e. poles inside unit circle) whose inverse 1/H(z) is also stable and causal (i.e. zeros inside unit circle)! Have some distortion that we want to compensate for: " All poles and zeros inside unit circle H(z) 1/H(z)! If H d (z) is min phase, easy: " H c (z)=1/h d (z) $ also stable and causal! Else, decompose H d (z)=h d,min (z) H d,ap (z) " H c (z)=1/h d,min (z) #H d (z)h c (z)=h d,ap (z) " Compensate for magnitude distortion 69 70 Generalized Linear Phase FIR GLP: Type I and II! An LTI system has generalized linear phase if frequency response can be expressed as:! Where A(ω) is a real function.! What is the group delay? 71 72 12
FIR GLP: Type III and IV Zeros of GLP System! FIR GLP System Function! If zero is on unit circle (r=1)! If zero is real and not on unit circle (θ=0) 73 74 FIR Design by Windowing FIR Filter Design! Given desired frequency response, H d (e jω ), find an impulse response! Obtain the th order causal FIR filter by truncating/windowing it 76 FIR Design by Windowing Tapered Windows 77 78 13
Tradeoff Ripple vs. Transition Width Optimality! Least Squares:! Variation: Weighted Least Squares: 79 80 Least-Squares Linear Phase Filter Least-Squares! Result will generally be non-symmetric and complex valued.! However, if is real, should have symmetry! 81 82 in-ax Ripple Design! Recall, is symmetric and real! Given ω p, ω s,, find δ, IIR Filter Design! Formulation is a linear program with solution δ,! A well studied class of problems 83 14
IIR Filter Design! Transform continuous-time filter into a discretetime filter meeting specs " Pick suitable transformation from s (Laplace variable) to z (or t to n) " Pick suitable analog H c (s) allowing specs to be met, transform to H(z) Bilinear Transformation! The technique uses an algebraic transformation between the variables s and z that maps the entire jω-axis in the s-plane to one revolution of the unit circle in the z-plane.! We ve seen this before impulse invariance 85 86 Transformation of DT Filters General Transformations H lp (Z) Z 1 = G(z 1 )! ap Z-plane#z-plane with transformation G 87 88 Discrete Fourier Transform Discrete Fourier Transform! The DFT! It is understood that, 90 15
DFT vs DTFT! Back to example 4 Circular Convolution! For x 1 [n] and x 2 [n] with length " Very useful!! (for linear convolutions with DFT) 91 92 Linear Convolution via Circular Convolution Block Convolution! Zero-pad x[n] by P-1 zeros! Zero-pad h[n] by L-1 zeros! ow, both sequences are length =L+P-1 93 94 Example of Overlap-Add L+P-1=16 Example of Overlap-Save L=11 y y L+P-1=16 P-1=5 Overlap samples y 95 96 16
Circular Conv. as Linear Conv. w/ Aliasing Example 2:! Let! Therefore x 3 p [n] = x 1 [n] x 2 [n] Linear convolution! The -point circular convolution is the sum of linear convolutions shifted in time by! What does the L-point circular convolution look like? 97 98 Example 2:! The L-shifted linear convolutions FFT 99 Fast Fourier Transform Decimation-in-Time FFT! Enable computation of an -point DFT (or DFT -1 ) with the order of just log 2 complex multiplications.! ost FFT algorithms decompose the computation of a DFT into successively smaller DFT computations. " Decimation-in-time algorithms " Decimation-in-frequency! Historically, power-of-2 DFTs had highest efficiency! odern computing has led to non-power-of-2 FFTs with high efficiency! Sparsity leads to reduce computation on order K log 101 3=log 2 ()=log 2 (8) stages 4=/2=8/2 multiplications in each stage 1 st stage has trivial multiplication 102 17
Decimation-in-Frequency FFT Spectral Analysis Using the DFT! 103 Spectral Analysis Using the DFT! " 104 Window Comparison Example y[n] = sin(2π 0.1992n) + 0.005sin(2π 0.25n) 0 n 128 Two important tools: " Steps for processing continuous time (CT) signals Applying a window # reduced artifacts Zero-padding # increases spectral sampling 105 Window Comparison Example 106 Time Dependent Fourier Transform!! Also called short-time Fourier transform To get temporal information, use part of the signal around every time point X [n, λ ) = x[n + m]w[m]e jλm m=!! 107 apping from 1D # 2D, n discrete, λ cont. Simply slide a window and compute DTFT 108 18
Time Dependent Fourier Transform Spectrogram Example 109 110 Final Project Final Exam! Due today @midnight Project must be submitted into Canvas " o late projects accepted! Final 4/30 " Location Towne 100 " Starts at exactly 3:00pm, ends at exactly 5:00pm (120 minutes) " Cumulative covers entire course " Except data converters, noise shaping (lec 11), adaptive filters (lec 22), wavelet transform compressive sampling (lec 24) " Closed book " Data/Equation sheet provided by me " 2 8.5x11 two-sided cheat sheet allowed " Calculators allowed, no smart phones " Old exams posted " Review session by Yexuan and Linyan on 4/26, 3-4pm in Towne 309 " Wednesday 4/25 last day for office hours " Piazza still available Penn ESE 570 Spring 2018 Khanna 111 Penn ESE 570 Spring 2018 Khanna 112 19