ELEN E4810: Digital Signal Processing Topic 2: Time domain 1. Discrete-time systems 2. Convolution 3. Linear Constant-Coefficient Difference Equations (LCCDEs) 4. Correlation 1
1. Discrete-time systems A system converts input to output: x[n] DT System y[n] E.g. Moving Average (MA): {y[n]} = f ({x[n]}) n A x[n] x[n-1] x[n-2] z -1 z -1 1/M 1/M + 1/M y[n] y[n] = 1 M M 1 k=0 (M = 3) x[n k] 2
Moving Average (MA) x[n] A x[n] x[n-1] x[n-2] z -1 z -1 1/M 1/M + 1/M y[n] x[n-1] x[n-2] -1 1 2 3 4 5 6 7 8 9-1 1 2 3 4 5 6 7 8 9 n n -1 1 2 3 4 5 6 7 8 9 n y[n] = 1 M M 1 k=0 x[n k] y[n] -1 1 2 3 4 5 6 7 8 9 n 3
MA Smoother MA smoothes out rapid variations (e.g. 12 month moving average ) e.g. signal noise x[n] = s[n] + d[n] y[n] = 1 5 4 k=0 x[n k] 5-pt moving average 4
Accumulator Output accumulates all past inputs: y[n] = n = n1 = x[] = x[] + x[n] = y[n 1]+ x[n] x[n] + y[n-1] z -1 y[n] 5
Accumulator x[n] A -1 1 2 3 4 5 6 7 8 9 n x[n] + y[n] y[n-1] z -1-1 1 2 3 4 5 6 7 8 9 n y[n-1] y[n] -1 1 2 3 4 5 6 7 8 9 n 6 M
Classes of DT systems Linear systems obey superposition: x[n] DT system y[n] if input x 1 [n] output y 1 [n], x 2 y 2... given a linear combination of inputs: then output for all Æ, Ø, x 1, x 2 i.e. same linear combination of outputs 7
Linearity: Example 1 Accumulator: y[n] = n = y[n] = x[n] = x 1 [n] + x 2 [n] x 1 [] + x 2 [] ( ) n = = ( x 1 []) + x 2 [] = x 1 [] + x 2 [] = y 1 [n] + y 2 [n] ( ) x[] Linear 8
Linearity Example 2: Teager Energy operator : y[n] = x 2 [n] x[n 1] x[n +1] x[n] = x 1 [n] + x 2 [n] y[n] = ( x 1 [n] + x 2 [n]) 2 ( x 1 [n 1] + x 2 [n 1] ) ( x 1 [n +1] + x 2 [n +1] ) y 1 [n] + y 2 [n] Nonlinear 9
Linearity Example 3: Offset accumulator: but y 1 [n] = C + y[n] = C + n = n = x 1 [] y 1 [n] + y 2 [n] y[n] = C + x 1 [] + x 2 [] ( ) n = Nonlinear.. unless C = 0 x[] 10
Property: Shift (time) invariance Time-shift of input causes same shift in output i.e. if x 1 [n] y 1 [n] then x[n] = x 1 [n n 0 ] y[n] = y 1 [n n 0 ] i.e. process doesn t depend on absolute value of n 11
Shift-invariance counterexample Upsampler: x[n] L y[n] y[n] = x[n/l] n =0, ±L, ±2L,... 0 otherwise y 1 [n] =x 1 [n/l] (n = r L) x[n] =x 1 [n n 0 ] y[n] =x[n/l] =x 1 [n/l n 0 ] = x 1 n L n 0 L Not shift invariant = y 1 [n L n 0 ] = y 1 [n n 0 ] 12
Another counterexample y[n] = n x[n] scaling by time index y 1 [n n 0 ] = ( n n ) 0 x 1 [n n 0 ] Hence If x[n] = x 1 [n n 0 ] then y[n] = n x 1 [n n 0 ] Not shift invariant - parameters depend on n 13
Linear Shift Invariant (LSI) Systems which are both linear and shift invariant are easily manipulated mathematically This is still a wide and useful class of systems If discrete index corresponds to time, called Linear Time Invariant (LTI) 14
Causality If output depends only on past and current inputs (not future), system is called causal Formally, if x 1 [n] y 1 [n] & x 2 [n] y 2 [n] Causal x 1 [n] = x 2 [n] n < N y 1 [n] = y 2 [n] n < N 15
Causality example Moving average: y[n] = 1 M M 1 k=0 y[n] depends on x[n-k], k 0 causal Centered moving average [ ] = y[ n + ( M 1) /2] y c n = 1 M.. looks forward in time noncausal.. Can make causal by delaying x[n k] ( x[n] + (M 1)/2 ) x[n k] + x[n + k] k=1 16
Impulse response (IR) Impulse ±[n] 1-3 -2-1 1 2 3 4 5 6 7 n (unit sample sequence) Given a system: if x[n] = ±[n] then y[n] = h[n] x[n] DT system y[n] LSI system completely specified by h[n] impulse response 17
Impulse response example x[n] Simple system: α 1 z -1 z -1 α 2 α 3 + y[n] x[n] 1-3 -2-1 1 2 3 4 5 6 7 n x[n] = ±[n] impulse y[n] -3-2 α 1 α 2α3-1 1 2 3 4 5 6 7 n y[n] = h[n] impulse response 18
2. Convolution Impulse response: Shift invariance: ±[n] LSI h[n] ±[n-n 0 ] LSI h[n-n 0 ] + Linearity: α ±[n-k] + β ±[n-l] LSI α h[n-k] + β h[n-l] Can express any sequence with ±s: x[n] = x[0]±[n] + x[1]±[n-1] + x[2]±[n-2].. 19
Convolution sum Hence, since For LSI, k= x[n] = x[k][n k] k= y[n] = x[k]h[n k] Convolution sum written as y[n] = x[n] * h[n] Summation is symmetric in x and h i.e. l = n k x[n] * h[n] = x[n l]h[l] = h[n] * x[n] l= 20
Convolution properties LSI System output y[n] = input x[n] convolved with impulse response h[n] h[n] completely describes system Commutative: Associative: x[n] h[n] = h[n] x[n] * * (x[n] * h[n]) * y[n] = x[n] * (h[n] * y[n]) Distributive: h[n] * (x[n] + y[n]) = h[n] * x[n] + h[n] * y[n] 21
Interpreting convolution Passing a signal through a (LSI) system is equivalent to convolving it with the system s impulse response x[n] h[n] y[n] = x[n] h[n] -3-2 y[n] = x[k]h[n k] = h[k]x[n k] k= -1 1 2 3 5 6 7 x[n]={0 3 1 2-1} h[n] = {3 2 1} n k= -3-2 -1 1 2 3 4 5 6 7 n 22
Convolution interpretation 1 k= y[n] = x[k]h[n k] Time-reverse h, shift by n, take inner product against fixed x y[n] 9 9 11 2 0-1 -3-2 -1 1 2 3 5 7 call h[-n] = g[n] n x[k] -3-3 -2 h[0-k] =g[k] -3-2 h[1-k] =g[k-1] -3-2 h[2-k] -2-1 1 2 3 5 6 7-1 1 2 3 4 5 6 7-1 1 2 3 4 5 6 7-1 1 2 3 4 5 6 7 k k k k 23
Convolution interpretation 2 k= y[n] = h[k]x[n k] Shifted x s weighted by points in h Conversely, weighted, delayed versions of h... -3-2 -1 1 2 3 4 5 6 7 k h[k] x[n] -3-3 -2 x[n-1] -2 x[n-2] -3-2 y[n] -1 1 2 3 5 6 7-1 1 2 3 4 6 7-1 1 2 3 4 5 7 9 9 11 2 0-1 -3-2 -1 1 2 3 5 7 n n n n 24
Matrix interpretation y[n] = x[n k]h[k] k= y[0] y[1] y[2]... = x[0] x[ 1] x[ 2] x[1] x[0] x[ 1] x[2] x[1] x[0]......... h[0] h[1] h[2] Diagonals in X matrix are equal 25
Convolution notes Total nonzero length of convolving N and M point sequences is N+M-1 Adding the indices of the terms within the summation gives n : k= y[n] = h[k]x[n k] k + n k ( ) = n i.e. summation indices move in opposite senses 26
Convolution in MATLAB The M-file conv implements the convolution sum of two finite-length sequences If a = [0 3 1 2-1] b = [3 2 1] then conv(a,b) yields [0 9 9 11 2 0-1] 27 M
Connected systems Cascade connection: Impulse response h[n] of the cascade of two systems with impulse responses h 1 [n] and h 2 [n] is h[n] = h 1 [n] * By commutativity, * 28
Inverse systems ±[n] is identity for convolution i.e. x[n] Consider * ±[n] = x[n] x[n] y[n] z[n] z[n] = h 2 [n] * y[n] = h 2 [n] * h 1 [n] * x[n] = x[n] if h 2 [n] * h 1 [n] = [n] h 2 [n] is the inverse system of h 1 [n] 29
Inverse systems Use inverse system to recover input x[n] from output y[n] (e.g. to undo effects of transmission channel) Only sometimes possible - e.g. cannot invert h 1 [n] = 0 In general, attempt to solve h 2 [n] * h 1 [n] = [n] 30
Inverse system example Accumulator: Impulse response h 1 [n] = μ[n] Backwards difference -3-2 -1 1 2 3 4 5 6 7 n.. has desired property: µ[n] µ[n 1] = [n] -3-2 -1 1 2 3 4 5 6 7 Thus, backwards difference is inverse system of accumulator. n 31
Parallel connection Impulse response of two parallel systems added together is: 32
3. Linear Constant-Coefficient Difference Equation (LCCDE) General spec. of DT, LSI, finite-dim sys: N d k y[n k] = p k x[n k] k=0 M k=0 defined by {d k },{p k } order = max(n,m) Rearrange for y[n] in causal form: N M d y[n] = k p y[n k] + k x[n k] d 0 d 0 k=1 k=0 WLOG, always have d 0 = 1 33
Solving LCCDEs Total solution y[n] = y c [n]+ y p [n] Complementary Solution satisfies N d k y[n k] = 0 k=0 Particular Solution for given forcing function x[n] 34
Complementary Solution General form of unforced oscillation i.e. system s natural modes Assume y c has form N d k nk = 0 k=0 nn N y c [n] = n ( d 0 N + d 1 N1 + + d N1 + d ) N = 0 d k Nk = 0 k=0 Characteristic polynomial of system - depends only on {d k } 35
Complementary Solution N d k Nk = 0 k=0 factors into roots λ i, i.e. ( 1 )( 2 )... = 0 Each/any λ i satisfies eqn. Thus, complementary solution: y c [n] = 1 n 1 + 2 n 2 + 3 n 3 +... Any linear combination will work α i s are free to match initial conditions 36
Complementary Solution Repeated roots in chr. poly: ( 1 ) L ( 2 )... = 0 y c [n] = 1 n 1 + 2 n n 1 + 3 n 2 n 1 +...+ L n L1 n 1 +... Complex λ i s sinusoidal y c [n] = i i n 37
Particular Solution Recall: Total solution y[n] = y c [n]+ y p [n] Particular solution reflects input Modes usually decay away for large n leaving just y p [n] Assume form of x[n], scaled by β: e.g. x[n] constant y p [n] = β x[n] = λ 0 n y p [n] = β λ 0 n (λ 0 λ i ) or = β n L λ 0 n (λ 0 λ i ) 38
LCCDE example y[n]+ y[n 1] 6y[n 2] = x[n] x[n] + y[n] Need input: x[n] = 8μ[n] Need initial conditions: y[-1] = 1, y[-2] = -1 39
LCCDE example Complementary solution: y[n]+ y[n 1] 6y[n 2] = 0; y[n] = n n2 ( 2 + 6) = 0 + 3 ( ) 2 ( ) = 0 roots λ 1 = -3, λ 2 = 2 y c [n] = ( 1 3) n + ( 2 2) n α 1, α 2 are unknown at this point 40
LCCDE example Particular solution: Input x[n] is constant = 8μ[n] assume y p [n] = β, substitute in: y[n]+ y[n 1] 6y[n 2] = x[n] + 6 = 8µ[n] 4 = 8 = 2 ( large n) 41
LCCDE example Total solution Solve for unknown α i s by substituting initial conditions into DE at n = 0, 1,... y[n]+ y[n 1] 6y[n 2] = x[n] n = 0 y[n] = y c [n]+ y p [n] = ( 1 3) n + ( 2 2) n + y[0]+ y[1] 6y[2] = x[0] 1 + 2 + +1+ 6 = 8 1 + 2 = 3 from ICs 42
LCCDE example n = 1 y[1]+ y[0] 6y[1] = x[1] 1 (3) + 2 (2) + + 1 + 2 + 6 = 8 2 1 + 3 2 =18 solve: α 1 = -1.8, α 2 = 4.8 Hence, system output: y[n] = 1.8(3) n + 4.8(2) n 2 n 0 Don t find α i s by solving with ICs at n = -1,-2 (ICs may not reflect natural modes; Mitra3 ex 2.37-8 (4.22-3) is wrong) 43 M
LCCDE solving summary Difference Equation (DE): Ay[n] + By[n-1] +... = Cx[n] + Dx[n-1] +... Initial Conditions (ICs): y[-1] =... DE RHS = 0 with y[n]=λ n roots {λ i } gives complementary soln y c [n] = n i i Particular soln: y p [n] ~ x[n] solve for βλ n 0 at large n α i s by substituting DE at n = 0, 1,... ICs for y[-1], y[-2]; y t =y c +y p for y[0], y[1] 44
LCCDEs: zero input/zero state Alternative approach to solving LCCDEs is to solve two subproblems: y zi [n], response with zero input (just ICs) y zs [n], response with zero state (just x[n]) Because of linearity, y[n] = y zi [n]+y zs [n] Both subproblems are fully realized But, have to solve for α i s twice (then sum them) 45
Impulse response of LCCDEs Impulse response: δ[n] LCCDE h[n] i.e. solve with x[n] = δ[n] y[n] = h[n] (zero ICs) With x[n] = δ[n], form of y p [n] = βδ[n] solve y[n] for n = 0,1, 2... to find α i s 46
LCCDE IR example e.g. y[n]+ y[n 1] 6y[n 2] = x[n] (from before); x[n] = δ[n]; y[n] = 0 for n<0 y c [n] = ( 1 3) n + ( 2 2) n yp[n] = βδ[n] 1 n = 0: y[0]+ y[1] 6y[2] = x[0] α 1 + α 2 + β = 1 n = 1: α 1 ( 3) + α 2 (2) + 1 = 0 n = 2: α1(9) + α2(4) 1 6 = 0 α 1 = 0.6, α 2 = 0.4, β = 0 thus n 0 h[n] = 0.6 3 ( ) n + 0.4( 2) n M Infinite length 47
System property: Stability Certain systems can be unstable e.g. x[n] + 2 z -1 y[n] y[n] -1 1 2 3 4 Output grows without limit in some conditions... n 48
Stability Several definitions for stability; we use Bounded-input, bounded-output (BIBO) stable For every bounded input x[n] < B x n output is also subject to a finite bound, y[n] < B y n 49
Stability example MA filter: y[n] = 1 M M 1 k=0 x[n k] y[n] = 1 M 1 M M 1 k=0 M 1 k=0 x[n k] x[n k] 1 M M B x B y BIBO Stable 50
Stability & LCCDEs LCCDE output is of form: y[n] = 1 1 n + 2 2 n +...+ 0 n +... αs and βs depend on input & ICs, but to be bounded for any input we need λ < 1 51
4. Correlation Correlation ~ identifies similarity between sequences: Cross correlation of x against y lag r xy [ ]= n= x[n]y[n ] Note: r yx [ ]= n= y[n]x[n ] call m = n = m= y[m + ]x[m] =r xy [ ] 52 M
Correlation and convolution Correlation: Convolution: r xy [n] = x[n] y[n] = k= k= x[k]y[k n] x[k]y[n k] Hence: r xy [n] =x[n] y[ n] Correlation may be calculated by convolving with time-reversed sequence 53
Autocorrelation Autocorrelation (AC) is correlation of signal with itself: Note: n= n= r xx [] = x[n]x[n ] r xx [0] = x 2 [n] = x = r xx [] Energy of sequence x[n] 54
Correlation maxima Note: Similarly: r xx [] r xx [0] r xx [] r xx [0] 1 r xy [] x y r xy [] r xx [0]r yy [0] 1 From geometry, x i 2 xy = x i y i = x y cos i when x//y, cosθ = 1, else cosθ < 1 angle between x and y 55
AC of a periodic sequence Sequence of period N: x [n] = x [n + N] Calculate AC over a finite window: M 1 r x x [] = x [n] x [n ] 2M +1 = 1 N N1 n=0 n=m x [n] x [n ] if M >> N 56
AC of a periodic sequence r x x [0] = 1 N N1 n=0 x 2 [n] = P x Average energy per sample or Power of x r x x [ + N] = 1 N N1 n=0 x [n] x [n N] = r x x [] i.e AC of periodic sequence is periodic 57
What correlations look like AC of any x[n] r xx [] AC of periodic Cross correlation r x x [] r xy [] 58
What correlation looks like 59
Correlation in action Close mic vs. video camera mic Short-time cross-correlation Dan Ellis 2013-09-11 60