Liear time ivariat systems Alejadro Ribeiro Dept. of Electrical ad Systems Egieerig Uiversity of Pesylvaia aribeiro@seas.upe.edu http://www.seas.upe.edu/users/~aribeiro/ February 25, 2016 Sigal ad Iformatio Processig Samplig 1
Liear time ivariat systems Liear time ivariat systems Fiite impulse respose filter desig Sigal ad Iformatio Processig Samplig 2
Fourier trasform ad covolutio Fourier trasform eables sigal ad iformatio processig Patters ad properties easier to discer o frequecy domai Also eables aalysis ad deig of liear time ivariat (LTI) systems Not altogether urelated to patter disceribility Two properties of LTI systems Characterized by their (impulse) respose to a delta iput Resposes to other iputs are covolutios with impulse respose Equivalet properties i the frequecy domai Characterized by frequecy respose = F(impulse respose) Output spectrum = iput spectrum frequecy respose Sigal ad Iformatio Processig Samplig 3
Systems A system is characterized by a iput (x()) output (y()) relatio This relatio is betwee fuctios, ot values Each output value y() depeds o all iput values x() x() System y() x() y() We ca, alteratively, cosider cotiuous time systems. The same. Sigal ad Iformatio Processig Samplig 4
Time ivariat systems A system is time ivariat if a delayed iput yields a delayed output If iput x() yields output y() the iput x( k) yields y( k) Thik of output whe iput is applied k time uits later x( k) System y( k) x() y() x( k) y( k) Sigal ad Iformatio Processig Samplig 5
Liear systems I a liear system iput a liear combiatio of iputs Output the same liear combiatio of the respective outputs I.e., if iput x 1 () yields output y 1 () ad x 2 () yields y 2 () Iput a 1 x 1 () + a 2 x 2 () yields output a 1 y 1 () + a 2 y 2 () a 1x 1() + a 2x 2() System a 1y 1() + a 2y 2() x 1() y 1() x 2() y 2() a 1x 1() + a 2x 2() a 1y 1() + a 2y 2() Sigal ad Iformatio Processig Samplig 6
Liear time ivariat systems Liear + time ivariat system = liear time ivariat system (LTI) Also called a LTI filter, or a liear filter, or simply a filter The impulse respose is the output whe iput is a delta fuctio Iput is x() = δ() (discrete time, δ(0) = 1) Output is y() = h() = impulse respose δ() System h() δ() h() Sigal ad Iformatio Processig Samplig 7
Scale ad shifted impulse resposes Sice the system is time ivariat (shift) Iput δ( k) Iduces output respose h( k) Sice the system is liear (scale) iput x(k)δ( k) Output x(k)h( k) Sice the system is liear (sum) x(k 1)δ( k 1) + x(k 2)δ( k 2) x(k 1)h( k 1) + x(k 2)h( k 2) δ() System h() δ() h() Sigal ad Iformatio Processig Samplig 8
Scale ad shifted impulse resposes Sice the system is time ivariat (shift) Iput δ( k) Iduces output respose h( k) Sice the system is liear (scale) iput x(k)δ( k) Output x(k)h( k) Sice the system is liear (sum) x(k 1)δ( k 1) + x(k 2)δ( k 2) x(k 1)h( k 1) + x(k 2)h( k 2) δ( k) System h( k) δ( k) h( k) Sigal ad Iformatio Processig Samplig 9
Scale ad shifted impulse resposes Sice the system is time ivariat (shift) Iput δ( k) Iduces output respose h( k) Sice the system is liear (scale) iput x(k)δ( k) Output x(k)h( k) Sice the system is liear (sum) x(k 1)δ( k 1) + x(k 2)δ( k 2) x(k 1)h( k 1) + x(k 2)h( k 2) δ( k) System h( k) x(k)δ( k) x(k)h( k) Sigal ad Iformatio Processig Samplig 10
Scale ad shifted impulse resposes Sice the system is time ivariat (shift) Iput δ( k) Iduces output respose h( k) Sice the system is liear (scale) iput x(k)δ( k) Output x(k)h( k) Sice the system is liear (sum) x(k 1)δ( k 1) + x(k 2)δ( k 2) x(k 1)h( k 1) + x(k 2)h( k 2) δ( k) x(k 1)δ( k 1) + x(k 2)δ( k 2) System h( k) x(k 1)h( k 1) + x(k 2)h( k 2) Sigal ad Iformatio Processig Samplig 11
Output of a liear time ivariat system Shift, Scale, ad Sum Is this a Covolutio? Of course Ca write ay sigal x as x() = + k= x(k)δ( k) Thus, output of LTI with impulse respose h to iput x is give by y() = + k= x(k)h( k) The above sum is the covolutio of x ad h y = x h Sigal ad Iformatio Processig Samplig 12
Output of a liear time ivariat system Theorem A liear time ivariat system is completely determied by its impulse respose h. I particular, the respose to iput x is the sigal y = x h. Iocet lookig restrictios Liearity + time ivariace Iduce very strog structure (aythig but iocet) x() h() (x h)() = x(k)h(t k) Ca derive exact same result for cotiuous time systems Sigal ad Iformatio Processig Samplig 13
Frequecy respose Frequecy respose = trasform of impulse respose H = F(h) Corollary A liear time ivariat system is completely determied by its frequecy respose H. I particular, the respose to iput X is the sigal Y = HX. X (f ) H(f ) Y (f ) = H(f )X (f ) Desig i frequecy Implemet i time Have doe this already, but ow we kow its true for ay LTI Sigal ad Iformatio Processig Samplig 14
Causality A causal filter is oe with h() = 0 for all egative < 0 Otherwise, we would respod to spike before seeig spike I geeral y() = + x(k)h( k) = x(k)h( k) k= k= The value y() is oly affected by past iputs x(k), with k If filter is ot causal but h() = 0 for all < N Make it causal with a delay h() = h( N) Frequecy respose of delayed filter H(f ) = H(f )e j2πfn Qualitatively the same filter Sigal ad Iformatio Processig Samplig 15
Fiite impulse respose A causal fiite impulse respose filter (FIR) is oe for which h() = 0 for all N We say the filter is of legth N; oly N values i h() are ot ull Ca write output at time as y() = h(0)x() + h(1)x( 1) +... h(n 1)x( N + 1) Ruig iput vector x N () = [x(); x( 1);... ; x( N + 1)] FIR filter vector respose h = [h(0), h(1),..., h(n 1)] Ca the write output at time as y() = h T x N Sigal ad Iformatio Processig Samplig 16
Fiite impulse respose filter desig Liear time ivariat systems Fiite impulse respose filter desig Sigal ad Iformatio Processig Samplig 17
Filter desig ad implemetatio We wat to utilize a LTI system to process discrete time sigal x() E.g., to smooth out the sigal x() show below x() h() H(f ) y() x() y() All LTIs are completely determied by their impulse resposes h Desig h ad implemet filter as time covolutio y = x h All LTIs are completely determied by their frequecy resposes h Desig H ad implemet filter as spectral product Y = HX Sigal ad Iformatio Processig Samplig 18
Frequecy desig ad time implemetatio Time ad frequecy represetatios are equivalet x() h() y() = (x h)() F F 1 F F 1 F F 1 X (f ) H(f ) Y (f ) = H(f )X (f ) Idetify patter trasformatio i frequecy domai Desig H Use iverse DTFT to compute impulse respose h = F 1 (H) Implemet covolutio i time y() = (x h)() Sigal ad Iformatio Processig Samplig 19
Causality ad ifiite respose Impulse respose h = F 1 (H) is typically ot causal ad ifiite E.g., Low pass filter with cutoff freq. W /2 H(f ) = W (f ) h() = fs /2 f s /2 H(f )e j2πfts df = W sic(πwt s ) H(f ) = F (f ) 1 F 1 F h() W /2 W /2 f - 3 W - 2 W - 1 W 1 W 2 W 3 W t Multiply by widow (chop) for fiite respose with N ozero coeffs. Delay h() to obtai a causal filter with h() = 0 for 0 Sigal ad Iformatio Processig Samplig 20
FIR filter desig Trasform h() ito fiite impulse respose h w () = h()w() F h w () Widow w() = 0 for / [N mi, N max ] Filter legth N = N max N mi + 1-3 W - 2 W - 1 W 1 W 2 W 3 W t Trasform h w () ito causal respose h w () = h w ( N mi ) F h w ( N mi ) Choose borders N mi ad N max to retai highest values of h() Ofte, aroud = 0. But ot always - 3 W - 2 W - 1 W 1 W 2 W 3 W t Sigal ad Iformatio Processig Samplig 21
Spectral effects of widowig ad delayig Multiplicatio i time domai Covolutio i frequecy domai As a result, istead of filterig with H(f ), we filter with H w = H W Choose widows with spectrum W = F(w) close to delta fuctio Time delay Multiplicatio with complex expoetial i frequecy H w (f ) = H w (f )e j2πfn mit s Irrelevat, as it should, we just shifted the respose Sigal ad Iformatio Processig Samplig 22
FIR filter desig methodology Procedure to desig time coefficiets of a FIR filter (1) Spectral aalysis to determie filter frequecy respose H(f ) (2) Iverse DFT (ot DTFT) to determie impulse respose h() (3) Determie r. of coefficiets N ad coefficiet rage [N mi, N max ] (4) Select widow w() Alters spectrum to H w = H W (5) Shift impulse respose by N mi time steps to make filter causal How to we use FIR filter coefficiets h() to implemet the filter? Sigal ad Iformatio Processig Samplig 23
FIR implemetatio The output y() of the FIR filter is give by the covolutio value y() = x(k)h( k) k= Sice h is fiite ad causal, oly N ozero terms. Make k = l N 1 y() = x(k)h( k)= h(l)x( l) k= (N 1) l=0 Easier to visualize whe writte i expaded form y() = h(0) x() + h(1) x( 1) +... + h(n 1) x( N + 1) The expressio above ca be implemeted with a shift register Sigal ad Iformatio Processig Samplig 24
Shift registers Upo arrival of sigal value x() we compute output value y() by Delay (shift) uits to shift elemets of sigal x Product (scale) uits to multiply with filter coefficiets x() Sum uits to aggregate the products h(k)x( k) x() T s x( 1) T s x( 2) T s x( 3) T s x( N +1) h(0) h(1) h(2) h(3) h(n 1) h(0)x() h(1)x( 1) h(2)x( 2) h(3)x( 3) h(n)x( N +1) Shift register ca be implemeted i hardware (or software) Sigal ad Iformatio Processig Samplig 25
Voice recogitio Spectral desig For a give word to be recogized we compare the spectra X ad X X Average spectrum magitude of word to be recogized X Recorded spectrum durig executio time 0.5 Average spectrum of spoke word oe 0.4 0.3 0.2 0.1 0-1.6-1.2-0.8-0.4 0 0.4 0.8 1.2 1.6 frequecy (KHz) Made copariso with ier product X T X Equivalet to usig X to filter X Y (f ) = H(f )X (f ) with H(f ) = X Sigal ad Iformatio Processig Samplig 26
Voice recogitio Filter desig (2) Impulse respose h() Iverse DFT of X (4) Widow to keep N = 1, 000 largest cosecutive taps 0.5 0 0.5 0.15 0.1 5 10 2 0 5 10 2 0.1 0.15 Sigal ad Iformatio Processig Samplig 27