052600 VU Signal and Image Processing Torsten Möller + Hrvoje Bogunović + Raphael Sahann torsten.moeller@univie.ac.at hrvoje.bogunovic@meduniwien.ac.at raphael.sahann@univie.ac.at vda.cs.univie.ac.at/teaching/sip/18s/
Discrete-time Systems Linear & Time Invariant (LTI) [Oppenheim + Schafer Ch. 2.2-2.3]
Systems Black-box Examples: Ideal delay system Moving average 3
Linearity Transformation, operator, system 4
Linear systems: examples Definition of system linearity: T ax 1 n + bx 2 n = at{x 1 n } + bt{x 2 [n]} 5
Linearity Input composed of multiple inputs Output is the same combination 6
Testing linearity 2003, JH McClellan & RW Schafer 7
Example: Testing Linearity Example 1 (Accumulator): 8
Example: Testing Linearity Example 2: 9
Time-invariant system Remain unchanged over time. If the same signal arrives at a later time the output would be the same 10
Time-invariance 11
Testing time-invariance 2003, JH McClellan & RW Schafer 12
Example: Testing time-invariance Example 1: Accumulator 13
Example: Testing time-invariance Example 2: 14
Impulse response LTI systems are characterized uniquely by their impulse response: Response of the system to the unit impulse 15
Impulse response Time-invariance implies: delayed impulse will cause delayed impulse response 16
Impulse response Linearity implies: linear combination of inputs causes the same linear combination of outputs Weighted combination of input The same weighted combination of output 17
Impulse response Arbitrary sequence: linear combination of delayed and weighted impulses Output sequence: Replace unit impulse by unit response 18
Impulse response Since any discrete-time signal is just a sum of scaled and shifted discrete-time impulses, we can find the output from knowing the input and the impulse response 19
LTI Filters FIR: Finite impulse response M = order of the filter (1+length of the imp. resp.) IIR: Infinite impulse response 20
FIR Filters 2003, JH McClellan & RW Schafer 21
LTI Systems are Convolvers Arbitrary sequence: linear combination of delayed and weighted impulses x n = x 0 δ n + x[1]δ n 1 + x 2 δ n 2 + x 3 δ n 3 Output seq: Replace unit impulse by unit response y n = x 0 h n + x 1 h n 1 + x 2 h n 2 + x 3 h n 3 Output: Discrete-time convolution of input with impulse resp. y n = m x m h[n m] LTI form y n = m h m x[n m] direct form 22
Convolution Sum y n = k= x k h[n k] y n = x n h[n] 23
Convolution in 1D Flip and slide form weighted average filter where one signal is flipped. Transient: input-on Steady-state behavior Transient: input-off 24
Convolution in 1D Values of interest? Input sequence of length L and filter order M The length of the output sequence: 25
Convolution in 1D: Example 2003, JH McClellan & RW Schafer 26
FIR Filters 2003, JH McClellan & RW Schafer 27
Convolution in 2D 28
2D Impulse response = filter mask Filter, mask, kernel, template, window Values= filter coefficients In 2D it is spatial filtering 29
Convolution in 2D: Example 30
Convolution Demo 1D Example in discrete domain http://pages.jh.edu/~signals/discreteconv2/index.html 1D Example in continuous domain https://phiresky.github.io/convolution-demo/ 2D Example: Image Kernels http://setosa.io/ev/image-kernels/ 31
Properties of convolutions Properties of LTI systems come directly from the properties of the convolution 32
Properties of LTI systems Convolution is commutative hk xk h n k hk xn k hk xk x k k k x[n] h[n] y[n] h[n] x[n] y[n] Convolution is distributive h 1 [n] x[n] h 2 [n] k h k h k xk h k xk h k x 1 2 1 2 + y[n] x[n] h 1 [n]+ h 2 [n] y[n] 33
Properties of LTI Systems Parallel realization of LTI systems Distributive property 34
Properties of LTI Systems Cascade realization of LTI systems Associative and commutative properties 35
LTI Building Blocks 2003, JH McClellan & RW Schafer 36
Quiz 2003, JH McClellan & RW Schafer 37
Causality of sequences n=0 start of the processing operations Causal: the most common type We turn on our signal generators Mixed: signal has existed even before we start processing 38
Properties of LTI Systems: Causality Impulse response: causal, anticausal or mixed System is causal if: h(n)=0, n<0 No output prior to input Non-causal filters can not be implemented in real time Have to wait for future signal values Examples: Interpolation or smoothing Not an issue with image processing 39
Examples of Causality Ideal delay can be both causal (n 0 >0) and anticausal (n 0 <0) h[n] = δ[n-n 0 ] Backward difference is causal h[n] = δ[n] δ[n-1] Forward difference is anticausal h[n] = δ[n+1] δ[n] Forward difference with one-sample delay in cascade Any FIR in cascade with sufficiently long delay can always be turned into a causual system. 40
Properties of LTI systems: Stability Bounded Input -> Bounded Output (BIBO) An LTI system is (BIBO) stable if and only if its impulse response is absolute summable k h k Important that the system is well-behaved Numerical stability in software. More important than causality 41
Properties of LTI systems: Stability Let s write the output of the system as hk xn k hk xn k y n If the input is bounded Then the output is bounded by k y n x[n] B B x x k h k k The output is bounded if the absolute sum is finite 42
Example: Test of Stability All FIR filters are stable For IIR, we have to test Example 1: h[n] = a n u[n], a <1 Example2: The accumulator 43
Inverse systems What is the overall impulse response? h n h i n = δ[n] 44
Linear Constant-Coefficient Difference Equations Subclass of LTI systems that can be represented with difference equations having constant coefficients y n = M i=1 a i y n i + Recursive difference equation => IIR filters The output is not uniquely specified for a given input The initial conditions are required Linearity, time invariance, and causality depend on the initial conditions If initial conditions are assumed to be zero (system at rest), the system is linear, time invariant, and causal 45 L i=0 b i x n i y n = a 1 y n 1 + a 2 y n 2 + + amy n M + b 0 x n + b 1 x n 1 + + blx[n L]
Example of Difference Equations Moving Average y[n] x[n] x[n 1] x[n 2] x[n 3] Difference Equation Representation 0 k 0 a k 3 k bkxn k where ak bk 1 y n k 0 FIR is a special case where recursive coefficients=0 46
Example of difference equations Accumulator: Difference Equation Representation 47
Difference equations Block diagram representation Example: Accumulator 48
FIR block diagram 2003, JH McClellan & RW Schafer 49
LTI Systems and Complex Exponentials Sinusoidal sequences carry amplitude and frequency x[n] = e jωn Let s see what happens if we feed x[n] into an LTI system: j(nk ) hk x n k hk e y n k k 50
LTI Systems and Complex Exponentials If x[n] is a complex exponential then y[n] is a complex exponential with the same frequency j(nk ) hk x n k hk e y n k k Complex exponentials are eigenfunctions of LTI systems jk jn j jn hk e e He e y n k eigenfunction eigenvalue The eigenvalue is called the frequency response of the system 51
LTI System Frequency Response The eigenvalue is called the frequency response of the system j jk hk e H e k H(e j ) is a complex function of frequency Has amplitude and phase Telling how the system response to a particular input frequency. Determines amplitude and phase change of the input Impulse response in time-domain <=> Frequency response in frequency-domain 52
LTI System Frequency Response Periodicity of Frequency Response H is periodic with period 2π. Since 53
Summary LTI: Linear and Time-Invariant Signal superposition property Mathematically equivalent descriptions or specifications of FIR/IIR filters: Impulse response h[n] Difference equations Processing = convolution operator ALL LTI systems have h[n] & use convolution Impulse response vs. Frequency response Filter design: Design of frequency-dependent signal manipulations 54