Multimedia Signals and Systems - Audio and Video Signal, Image, Video Processing Review-Introduction, MP3 and MPEG2 Kunio Takaya Electrical and Computer Engineering University of Saskatchewan December 19, 2007 ** Go to full-screen mode now by hitting CTRL-L 1
Contents 1 1D Signal Processing - FIR Filters 3 2 1D Signal Processing - IIR Filters 17 3 1D Signal Processing - DFT & FFT 33 4 continue to 2D Signal Processing 54 2
1 1D Signal Processing - FIR Filters 1. FIR (Finite Impulse Response) Filters The general form of digital filters, and that of FIR with b i = 0 for i = 0,, N; H(z) = N a i z i i=0 1 + Input-output relationship is; = N b i z i i=0 N a i z i i=0 Y (z) = H(z)X(z) in z-transform y(n) = a 0 x(n) + a 1 x(n 1) +... + a N x(n N) in time domian 3
Impulse response, i.e. x(0) = 1, x(n) = 0 for n = 1,, is zero after time n = N. Linear phase means delay of a fixed time, H(e jω ) = ejω(t T ) e jωt e jωt = 1 ( ωt ) Frequency response of digital filters given by y(n) = a 0 x(n)+a 1 x(n 1) +a N x(n N) b 1 y(n 1) b N y(n M) is obtained from that sine input gives sine output; x(n) = e jωn, y(n) = Ae jωn e jθ a 0 + a 1 e jω + a 2 e jω2 + a 1 e jωn = Ae jθ (1 + b 1 e jω + b 2 e jω2 + b 1 e jωm ) 4
Thus, frequency response (amplitude and phase) is given by Ae jθ = H(z)j z=e jω = H(e jω ) 2. Symmetric FIR Filters When FIR filters are symmetric, h(n) = h(n 1 n), for N is odd, H(e jω ) = = = (N 3)/2 n=0 (N 3)/2 n=0 h(n)e jωn + h( N 1 jω )e 2 + 2 n=(n+1)/2 h(n)fe jωn + e jω( n) g + h( N 1 jω )e 2 2 (N 3)/2 = e jω()/2 [ n=0 h(n) cosfω(n N 1 2 h(n)e jωn )g + h( N 1 )] 2 5
If N is even, the frequency response is given by N/2 1 H(e jω ) = e jω()/2 [ n=0 h(n) cosfω(n N 1 )g] 2 Example: When a FIR filter is symmetrical h(n) = h(n 1 n), N = 5 and h(0) = h(1) = h(2) = 1 5, the frequency response of this FIR filter is, H(e jω ) = 0.4e j2ω [cos(2ω) + cos(ω) + 0.5]. 3. Anti-symmetric FIR Filters If N is odd, the frequency response is (N 3)/2 H(e jω ) = e jω()/2 e jπ/2 n=0 2h( N 1 2 n) sin(ωn) 6
If N is even, the frequency response is N/2 1 H(e jω ) = e jω()/2 e jπ/2 n=0 2h( N 1 2 n) sin(ωn) Anti-symmetric filters do not satisfy linear phase and they are used for high-pass filtering, a typical example is difference filters. 4. FIR Filter Design - the Window Method The frequency response of discrete systems is periodic, therefore, can be expanded into the Fourier series. H(z)j z=e jω = + n= c n e jωn 7
The coefficients are calculated by c n = 1 2π +π π H(jω)e jωn dω The infinite sequence c n is expressed in the z-transform as H(z); H(z) = + c n z n The window method applies a window w(n) to a delayed sequence c n α to approximate the infinite sequence c n with a N-point sequence. h(n) = c n α w(n), n = 0, 1, 2,, N 1 To design a linear phase LP (low pass) filter, let ω c be a cut-off frequency, and the window function w(n) be a rectangular window, i.e. w(n) = 1 for 0 n N 1. The ideal frequency response that 8
incorporates the delay α is, H(e jω ) = e jωα for 0 ω < ω c 0 for ω c < ω π The Fourier series coefficients c n that takes the delay α into account is, c n = 1 2π +ωc ω c e jωα e jωn dω = 1 1 2π j(n α) ejω(n α) j +ω c ω c sin ω c (n α) if n α = π(n α) ω c if n = α π 9
Let α = (N 1)/2 regardless of N even or odd, we have sin ω c (n N 1 ) 2 c n = π(n N 1 if n (N 1)/2 ) 2 ω c if n = (N 1)/2 π which determines the filter sequence h(n) = c n w(n). 5. LP, HP, BP and BS filters LP, HP, BP and BS filters before applying a window function are given as follows. The filter is of linear phase, and α = (N 1)/2. Cut-off frequencies are ω 1 and ω 2 (> ω 1 ). 1. Low-pass: h LP (n) = sin ω 1(n α) π(n α) 10
2. High-pass: 3. Band-pass: 4. Band-stop: h BS (n) = h HP (n) = h LP (α) = ω 1 π h HP (α) = ω 1 π sin π(n α) π(n α) sin ω 1(n α) π(n α) h BP (n) = sin ω 2(n α) sin ω 1 (n α) π(n α) h BP (α) = ω 2 ω 1 π sin π(n α) π(n α) sin ω 2(n α) sin ω 1 (n α) π(n α) 11
h BS (α) = ω 2 ω 1 π A FIR band-pass filter design is shown. Specs are N = 64, f 1 /f s = 0.2 and f 2 /f s = 0.3. Hamming windows is used. 12
6. Various Window Functions When two time domain discrete sequences, x(i) and w(i) of a length N, are multiplied, z(i) = x(i)w(i) it is equivalent in the frequency domain to the periodic convolution. Ffx(n)w(n)g = 1 2π +π π X[j(ω u)]w (ju)du The spectrum of the data x(i) is periodic because of the discreteness. The spectrum X(e jω ) is continuous in frequency, which is smeared by the convolution (referred to as window effect). The spectrum of the rectangular discrete window D(jω) is called Derichlet kernel and it is the Fourier transform of N sampling delta 13
functions spaced with T = 1. D(jω) is a periodic function. D(jω) = sin( Nω 2 ) sin( ω e jω 2 2 ) Rectangular Window For a rectangular window, w(n) = 1, n = 0, 1,..., N-1, W (jω) = D(jω) Bartlett Window (Triangular Window) w(n) = 2n N 1 2 2n N 1 for 0 n 2 for 2 < n N 1 14
W (jω) = D(jω) 2 N 2 ejω N 2 Hamming/Hanning Windows w(n) = α (1 α)cos( 2πn N 1 ) where α = 0.54 for Hamming window, and α = 0.5 for Hanning window. W (jω) = D(jω) 0.5(1 α)[d(j(ω 2π N 1 ))+D(j(ω+ 2π N 1 ))] Adjusting α from 0.5 of Hanning window to 0.54 of Hamming window has the effect to diminish the second lobe of Hanning window. 15
The frequency response of the Hamming window, and the Hanning window are shown with time domain window profiles. 16
2 1D Signal Processing - IIR Filters 7. IIR (Infinite Impulse Response) Filters IIR filters are not linear phase, which means that waveform will be distorted. IIR filters can realize the desired frequency response with a much shorter filter length (order) than FIR filters. IIR filters are usually converted from analog prototype filters. The design procedure follow the following steps. 1. Choose an analog filter type out of (a) Butterworth Filter (b) Bessel Filter (c) Chebyschev Filter (d) Elliptic Filter 2. Determine the order of a filter 17
3. s-domain frequency band transformation (a) LP to LP (b) LP to HP (c) LP to BP (d) LP to BS 4. Frequency Prewarping if bilinear transformation is used in the next step. 5. Transformation from s-domain to z-domain (a) Impulse Invariant Transformation (b) Bilinear Transformation (c) Matched z-transform method 6. z-domain frequency band transformation if frequency band transformation was not done in s-domain 18
8. Butterworth Filters The LP (Low Pass) Butterworth filters have a following magnitude squared frequency response. jh(jω)j 2 = 1 ( Ω 1 + Ω c ) 2N Where, Ω c is a cut-off frequency, and N is the order of the Butterworth filter. The poles of the Butterworth filter are found by 1 + ( Ω Ω c ) 2N = 1 + ( s jω c ) 2N = 0 From this characteristic equation, s Ω c = j( 1) 1/2N = e jπ 2 ( 1) 1/2N 19
Let s/ω c = e jθ to express s in the polar form, Thus, e j(θ π/2)2n = 1 = e jπ(2k+1) for k = 0, 1, 2, N 1 θ = π 2k + 1 (1 ± ) for k = 0, 1, 2, N 1 2 N The poles of the LP Butterworth filter are located on a circle of a radius Ω c and at angles θ. Transfer function of the Butterworth filter of order N = 2 is, θ = π/4, 3π/4, 5π/4, 3π/4, 7π/4 and π/4, for N = 2 Choosing ±3π/4, H(s) = = 1 (s + e j3π/4 )(s + e j3π/4 ) 1 (s + 1/ 2 j/ 2)(s + 1/ 2 + j/ 2) 20
= 1 s 2 + 2s + 1 Transfer function of the Butterworth filter of order N = 3 is, Choosing ±2π/3, and π, θ = 0, ±π/3, ±2π/3, ±pi, for N = 3 H(s) = 1 s 3 + 2s 2 + 2s + 1 The Butterworth polynomials from the 4th to 7th order are, N = 4: (s 2 + 0.765s + 1)(s 2 + 1.848s + 1) N = 5: (s + 1)(s 2 + 0.618s + 1)(s 2 + 1.618s + 1) N = 6: (s 2 + 0.518s + 1)(s 2 + 1.141s + 1)(s 2 + 1.932s + 1) N = 7: (s + 1)(s 2 + 0.445s + 1)(s 2 + 1.247s + 1)(s 2 + 1.802s + 1) 21
9. Analog Frequency Band Transformation Consider a 1st order LP prototype filter, H(s) = 1 s + 1. (1) Lowpass (LP) to LP Transformation If a desired cut-off frequency is Ω c instead of Ω = 1, the transformation is s s Ω c H LP = 1 s/ω c + 1 = Ω c s + Ω c This transfer function obviously has the cut-off frequency at Ω c and the DC gain 1. (2) LP to HP (High-pass) Transformation 22
s Ω c s H HP = 1 Ω c /s + 1 = = 1 H LP (s) s s + Ω c where Ω c is the desired cut-off frequency. Letting s = jω = 0, we see that this filter have no DC response. But gain is 1 at ω =. Therefore, the transformed transfer function is of HP. (3) LP to BP (Band-pass) Transformation Band-pass filters can not be realized from one 1st order system. The simplest BP filter is a 2nd order system of the following general form. H BP (s) = As s 2 + Bs + C Now, make the transfer function have its centre frequency 23
equal to the mid-point between the upper cut-off frequency Ω 2 and the lower cut-off frequency Ω 1 in the log scale, and the gain be 1. At the two cut-off frequencies, the gain should drop down to -3 db, or equivalently 1/ 2 of the mid frequency gain. The centre frequency is Ω 0 = Ω 1 Ω 2 found from From the gain conditions, log Ω 0 = 1 2 (log Ω 1 + logω 2 ) jh BP (Ω 0 )j = 1, and jh BP (Ω 1 or Ω 2 )j = 1/ 2 We find that A = B = Ω 2 Ω 1 and C = Ω 1 Ω 2. Substituting A, B and C in the 2nd order BP filter transfer function, H BP (s) = Ω 2 Ω 1 )s s 2 + (Ω 2 Ω 1 )s + Ω 1 Ω 2 24
= 1 s 2 + Ω 1 Ω 2 (Ω 2 Ω 1 )s + 1 This indicates that the prototype transfer function can be transformed into a BP filter by s s2 + Ω 1 Ω 2 (Ω 2 Ω 1 )s (4) LP to BS (Band-stop) Transformation Applying H BS (s) = 1 H BP (s), s (Ω 2 Ω 1 )s s 2 + Ω 1 Ω 2 The BS filters are also 2nd order, if the prototype is a 1st order system. 25
10. Order of the Butterworth Filters Desired cut-off frequency of the low pass filter is f c. At a frequency f > f c in the stop-band, the gain should be less than -3dB. We let the stop band gain be less than A db < 0dB in all frequencies greater than f 1. Find the order N of the Butterworth low pass filter. Assume that the filter is The amplitude squared response of the Butterworth filters is jh(jω)j 2 = 1 ( Ω 1 + Ω c ) 2N Since jh(jω)j is a monotonously decreasing function, the gain at f = f 1 should be less than A db. Applying frequency prewarping 26
to f c and f 1, Solving for N, A 10 log N 1 2 1 + log 1 tan πf 1 f s tan πf c f s log(10 A/10 1) tan πf 1 f s tan πf c f s The smallest integer N satisfying the above inequality is the order of the Butterworth filter. 2N 27
11. Bilinear Transformation A numerical integration by the trapezoidal approximation is, y(k) = y(k 1) + T 2 fx(k) + x(k 1)g The integrated value y(k) up to the kth sample point is the sum of the integration up to the (k 1)th point, and the trapezoidal area calculated by (x(k) + x(k 1))T/2. Taking the z-transform, Y (z) = z 1 Y (z) + y( 1) + T 2 fx(z) + z 1 X(z) + x( 1)g Assuming x( 1) = 0, and y( 1) = 0, Y (z) X(z) = T 2 = T 2 1 + z 1 1 z 1 z + 1 z 1 28
The Laplace transform of integration is, Y (s) X(s) = 1 s Since two expressions represent integration, those are now equated. s = 2 T z 1 z + 1 This is a mapping function which relates the s-plane and the z-plane. The frequency response in the z-transform domain is determinded by z = e jω. s = 2 z 1 T z + 1 j z=e jω = j 2 sin ω/2 T cos ω/2 = j 2 T tan ω 2 29
s = σ + jω, therefore, Ω = 2 T tan ω 2 If ω means the actual frequency (not bewtween ±π), we need to normalize by f s = 1 T. Rewrite the above equation with s = ejωt as Ω = 2 T tan ωt 2. The range of π < ωt < +π spans < Ω < + in s-plane. The digital frequency ω and the analog frequency Ω are not linearly related. Reversing z and s, Substituting s = σ + jω, we have z = (2/T ) + s (2/T ) s jzj 2 = (2/T + σ)2 + Ω 2 (2/T σ) 2 + Ω 2 30
If s is in the left half of the s-plane, i.e. σ < 0, then it is clear that jzj < 1. Therefore, all the stable poles in the s-domain are mapped inside the unit circle in the z-plane where poles are stable. 31
12. Frequency Prewarping A merit of introducing the bilinear transformation is a simple means to convert a s-domain transfer function into the z-domain. But, analog frequency Ω and digital frequency ω are not related linearly. If an analog filter has a cut-off frequency of 100 Hz, 2π 100Hz = 2 T tan ωt 2 its corresponding digital frequency f is 96.89Hz, if T =1ms. If 100Hz is desired in the digital filter, it is necessary to set the cut-off frequency of the analog filter slightly higher than what is desired in the digital filter. The equation for frequency prewarping is, Ω = 2 T tan ωt 2 = 2f s tan ω 2f s 32
3 1D Signal Processing - DFT & FFT 13. Discrete Fourier Transform (DFT) In DFT we deals with a restricted set of data such that 1. Time data sequence x(i) is real data, not complex. - this condition is later removed, though. 2. Time data sequence x(i) is periodic of a length N. 3. Since signal is periodic, its spectrum is discrete (harmonics). Let X(m) be m-th harmonic. 4. X(m) is band-limited to include only up to N/2-th harmonic. Having these conditions satisfied, the Inverse Discrete Fourier Transform, IDFT, is given first as follows. IDFT calculates the time sequence x(i), i = 0 N 1 from given spectrum samples 33
(harmonics) X(n), n = 0 N 1. x(i) = 1 N = 1 N N 2 n= N 2 +1 X(n)e j 2π n=0 N ni X(n)e j 2π N ni (1) The Discrete Fourier Transform, DFT, can be derived from the IDFT. i=0 x(i)e j 2π N mi = = [ 1 N i=0 n=0 n=0 X(n)[ 1 N e j 2π N ni ]e j 2π N mi i=0 e j 2π N (n m)i ] 34
= n=0 = X(m) X(n)δ(n m) With help of Kronecker s delta function δ(n m), we showed that the mth discrete spectrum X(m) is calculated from the time sequence of x(i). The discrete Fourier transform and its inverse are: DFT: X(m) = i=0 IDFT: x(i) = 1 N x(i)e j 2π N mi n=0 X(n)e j 2π N ni 35
14. Aliasing (Spectral Folding) Let s do a bit of math here, as an exercise of DFT. Consider a complex sinusoid of a frequency ω a higher than sampling frequency ω s = 2πf s. Let the number of samples is N. This sets the period of fundamental harmonic to be T = N (1/f s ), and the fundamental frequency is ω 0 = 2π/T = 2π/(N t). Let ω a = kω s + lω 0 Then, the complex sinusoid is x(i) = e j(kω s+lω 0 )t = e j(kω s+lω 0 ) t i t= t i 2π 2π j(k = e t +l N t ) t i = e j(k2πi+l 2π N i) 36
Calculate the DFT, X(m) = = = = i=0 i=0 i=0 i=0 x(i)e j 2π N mi e j(k2πi+l 2π N i) e j 2π N mi e jl 2π N i e j 2π N mi e j 2π N (l m)i = Nδ(m l) Thus, we see that the frequency higher than ω s is shifted down to the baseband, i.e. from ω a = kω s + lω 0 down to lω 0. This is aliasing or spectral folding in case of real signals. 37
Nyquist Sampling Theorem When analog signal is sampled, the sampling frequency ω s must be chosen greater than the Nyquist frequency, which is twice as high as the highest frequency of the signal. Another Aspect Analog signal can be reconstructed from discrete samples, if band limited to ω s /2. f(t) = n= f( nπ ) sin(ω st nπ) ω s ω s t nπ 38
15. DFT of Real Signals The discrete Fourier transform X(m) is given by X(m) = i=0 x(i)e j 2π N mi m = 0, 1,..., N 1 Consider the spectrum X(N m), when x(i) is a real sequence. X(N m) = = i=0 i=0 = X(m) x(i)e j 2π N (N m)i x(i)e +j 2π N mi Real part of real signals Imag part of real signals is symmetric (even). is anti-symmetric (odd). 39
X(N m) = X(N m + kn) since X(m) is periodic. = X( m) if k = 1 This means that the upper half of DFT represents the spectrum of negative frequencies. X(0) is DC component. X(1) is the fundamental harmonic. N/2 is the highest frequency component resolvable by DFT for N (even), or (N 1)/2 for N (odd). 40
1 Square wave Real Part 20 Imag. Part 0.9 30 15 0.8 25 10 0.7 0.6 20 5 0.5 15 0 0.4 5 0.3 10 10 0.2 0.1 5 15 0 0 20 40 60 0 0 20 40 60 20 0 20 40 60 41
FFT magnitude FFTshift magnitude 30 30 25 25 20 20 15 15 10 10 5 5 0 0 20 40 60 0 0 20 40 60 42
16. Shifting DFT to center at DC It is often more convenient to bring the DC component to the centre of DFT range instead of having the highest frequency in the middle. This is called fftshift in MATLAB. This shift can be accomplished by introducing another sequence y(i). y(i) = ( 1) i x(i) Taking DFT of the new sequence y(i), Y (m) = = = i=0 i=0 i=0 y(i)e j 2π N mi x(i)( 1) i e j 2π N mi x(i)e ±jπi e j 2π N mi 43
= i=0 x(i)e j 2π N (m N 2 i) = X(m N 2 ) Thus, multiplying a sequence x(i) by ( 1) i will shift the result to N/2. This concept is also useful in two-dimensional DFT to bring lower frequency components to the centre of the spectral plane. For two-dimensional DFT of x(i, j), X(m, n) = i=0 j=0 x(i, j)e j 2π N mi e j 2π N nj. Modifying a 2D data x(i, j) by y(i, j) = ( 1) i+j x(i, j) moves the two-dimensional spectrum Y (m, n) = DFT y(i, j) to center at m = N 2 and n = N 2. 44
45
17. DFT of Symmetric waveforms The DFT of x(i) is, X(m) = i=0 x(i) cos( 2π N mi) + j i=0 x(i) sin( 2π N mi) Assuming that x(i) is a symmetrical sequence, i.e. x(n i) = x(i), consider the second term of this equation. i=0 = = x(i) sin( 2π N mi) N 2 1 i=0 N 2 1 i=0 x(i) sin( 2π N 2 N mi) + i=1 x(i) sin( 2π N 2 N mi) + i=1 x(n i) sin( 2π N x(i) sin( 2π N mi) m(n i)) 46
= 0 Hence, the DFT of the symmetrical x(i) is real. In the case of anti-symmetrical sequences, x(n i) = x(i), we consider the first term of the above equation instead. i=0 x(i) cos( 2π N mi) = 0 Hence, the DFT of the anti-symmetrical x(i) is imaginary. 47
18. Decimation in Time FFT DFT is calculated according to the DFT formula given by X(k) = n=0 2πnk j x(n)e N k = 0, 1,..., N 1 In Fast Fourier Transform (FFT), data length N is chosen a power of 2, N = 2 P. Let, W N = e j 2π N. Using this notation W N, X(k) = = n=0 N/2 1 n=0 x(n)w kn N x(2n)w 2nk N + N/2 1 n=0 x(2n + 1)W (2n+1)k N 48
Extract even numbered terms and odd numbered terms of x(n), then define, x e (n) = x(2n) x o (n) = x(2n + 1) n = 0, 1, N 2 1 Since W 2 N = (ej2π/n ) 2 = e j2π/(n/2) = W N/2, X(k) = N/2 1 n=0 x e (n)w kn N/2 + W k N = X e (k) + WNX k o (k) n = 0, 1, N 2 1 N/2 1 n=0 x o (n)w kn N/2 49
When k N/2, we write k as k = i + N/2 for i = 0, 1, N 2 1. x(i+n/2) = Since, N/2 1 n=0 N/2 1 x e (n)w n(i+n/2) N/2 +W i+n/2 N x o (n)w n(i+n/2) N/2 n=0 W n(i+n/2) N/2 = W ni N/2 W (i+n/2) N = W i N The DFT X(k) for k greater than or equal to N/2 is given by, X(k) = X(i + N/2) = X e (i) W i NX o (i) Thus, we can write the butterfly equation, x(k) = 1 W N k x(k + N/2) 1 WN k X e(k) X o (k) 50
= 1 1 1 1 k = 1, 2, N 2 1 X e (k) W k N X o(k) A block daigam of the decimation in time 8 point FFT is shown below. In the decimation in time FFT, input sequence x(n) needs to be shuffled to make the sequence arranged in the bit-reversed order of which is (000), (001), (010), (011), (100), (101), (110), (111) (000), (100), (010), (110), (001), (101), (011), (111). 51
52
(000), (001), (010), (011), (100), (101), (110), (111) The number of multiply and add operation involved in the ordinary DFT calculations is 2 N. That of FFT is N log 2 N. Where, log 2 N is the number of stages that FFT progresses from time data x(n) (left) to spectrum X(k) (right) in the block diagram. 53
4 continue to 2D Signal Processing 54