Multirate signal processing

Multirate signal processing Discrete-time systems with different sampling rates at various parts of the system are called multirate systems. The need for such systems arises in many applications, including digital audio, communication, image processing, and HDTV. Conceptually, the process of sampling rate conversion consists of two steps: 1) reconstruction of a continuous-time signal, and 2) resampling at a different rate. In practice, however, the conversion is implemented by means of discrete-time signal processing. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 1/38

Downsampling by an integer factor A sampling rate compressor or simply compressor is a systems that reduced the sampling rate of an input discrete-time signal by an integer factor of D. The spectrum of the sampled discrete-time signal x D [m] is given by X D (e jωt D ) = 1 X c (j(ω lω D )) = T D l Z = 1 DT l Z ( X c (j Ω l 2π ) ), DT where Ω D = 2π/T D = Ω s /D is the reduced sampling rate. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 2/38

Downsampling by an integer factor (cont.) Expressing l = m + kd, with m = 0, 1,..., D 1, we obtain X D (e jωt D ) = 1 D D 1 m=0 [ 1 T ( X c (j Ω m 2π DT k 2π )) T k Z }{{} X(e j(ω 2πm/DT )T ) We note that the above observation is true since 1 ( X c (j Ω k 2π )) = X ( e jωt ). T T k Z Consequently, we obtain X D (e jωt D ) = 1 D D 1 m=0 ( ) X e j(ω mω D)T, which relates directly the spectra of x[n] and x D [n]. ]. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 3/38

Downsampling by an integer factor (cont.) If we define the normalized frequency variable as ω = ΩT D, and note that ΩT = ΩT D /D = ω/d, we obtain X D (e jω ) = 1 D D 1 m=0 X (e j(ω 2πm)/D) Hence, the process of computing X D (e jω ) directly from X(e jω ) consists of: 1 Stretching X(e jω ) by a factor of D to obtain X(e jω/d ); 2 Placing D copies of X(e jω/d ) at the frequencies ω = 2πm, where m = 0, 1,..., D 1; 3 Adding the D stretched and shifted replicas and then dividing the result by D to obtain X D(e jω ). Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 4/38

Example: D = 3 In the case of downsampling by a factor of D = 3, we have X D (e jω ) = 1 [ ] X(e jω/3 ) + X(e j(ω/3+2π/3) ) + X(e j(ω/3+4π/3) ). 3 Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 5/38

Sampling rate conversion To establish a closer analogy between CT and DT sampling, we define the discrete-time sampling rate as f s = 1/D or ω s = 2π/D, which simply means keep one out of every D samples. Then, to avoid aliasing, we should have the following conditions If X(e jω ) = 0, ω H ω π then ω s = 2π D 2ω H. If the above condition is not satisfied, we will have aliasing distortion. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 6/38

Sampling rate conversion (cont.) To avoid aliasing distortion we precede the downsampling by a lowpass filter with cutoff frequency ω c = π/d. The combined system is called a decimator and the corresponding process is known as decimation. If we use an FIR lowpass filter of order M, the output of the decimator is given by x D [m] = v[md] = M h[k]x[md k]. k=0 How is it different from convolution? Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 7/38

Example: D = 2 Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 8/38

Back reconstruction In the absence of aliasing, x c (t) can be recovered from x D [m] according to x c (t) = m Z x D [m] Evaluating x c (t) at t = nt results in x[n] = m Z sin [π(t mdt )/DT ]. π(t mdt )/DT sin [π(n md)/d] x D [m] π(n md)/d, which provides perfect reconstruction of x[n] directly from x D [m]. Clearly, the ideal discrete-time interpolator is not practically realizable, which necessitates the use of approximations. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 9/38

Upsampling by an integer factor Increasing the sampling rate of x[n] by an integer factor I requires the insertion of (I 1) samples between consecutive samples of x[n]. Thus, upsampling is an information preserving operation. The interpolated signal x I [n] can be obtained using the formula x I [n] = m Z x[m] sin [π(n mi)/i], π(n mi)/i which is a special case of a more general interpolation formula x I [n] = m Z x[m]g r [n mi], where g r [n] is an interpolating filter (aka interpolator ). Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 10/38

Upsampling by an integer factor (cont.) In the domain of DTFT, x I [n] = m Z x[m]g r[n mi] becomes X I (e jω ) = G r (e jω )X(e jωi ), where X(e jωi ) is the DTFT of sequence x u [n] obtained by inserting (I 1) zeros between consecutive samples of x[n], that is, by { x[n/i], n is a multiple of I, x u [n] = 0, otherwise, with X u (e jω ) = X(e jωi ). The above system is called upsampler or sampling rate expander. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 11/38

Upsampling by an integer factor (cont.) Thus, the interpolation can be alternatively implemented as X I (e jω ) = G r (e jω )X u (e jω ), which corresponds to an LTI system described by x I [n] = x u [m]g r [n m]. m Z Schematically, we have Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 12/38

Example: I = 3 In practice, the ideal interpolators are replaced by their realizable approximations. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 13/38

Example: Linear interpolation Due to its computational efficiency, linear interpolation has long been a default choice of many multirate systems. Let m and n = mi + k (with k = 0, 1,..., I 1) be a slow and a fast time index, respectively. Then, linear interpolation is defined as x I [(m 1)I + k] = x[m 1] + (x[m] x[m 1]) (k/i), for k = 0, 1,..., I 1. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 14/38

Example: Linear interpolation (cont.) It is easy to verify that, in this case, x I [n] is obtained by convolving x u [n] with the triangular interpolator { 1 n /I, I < n < I, g lin [n] = 0, otherwise. The frequency response of the linear interpolation filter is given by [ ] 2 sin(ωi/2). G lin (e jω ) = 1 I ω/2 Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 15/38

Example: Linear interpolation (cont.) Despite the limited ability of linear interpolation to cope with aliasing, its performance is often deemed satisfactory. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 16/38

Sampling rate change by a rational factor Interpolation and decimation can be combined to change the sampling rate by a rational factor I/D. Let s consider the cascade connection of an interpolator and a decimator. The effective sampling rate of the output signal is T 0 = T D/I. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 17/38

Sampling rate change by a rational factor (cont.) Since the interpolation and decimation filters are in cascade and operate at the same sampling rate, they can be replaced by a single lowpass filter H(e jω ) = { I, 0 ω ω c 0, ω c ω π { π where ω c = min I, π }. D When I > D (sampling rate increase), H(e jω ) acts as a postfilter to remove the replicas introduced by the upsampling operation. When I < D (sampling rate decrease), H(e jω ) acts as an antialiasing filter for the downsampling operation. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 18/38

Half-band FIR filters Decimation and interpolation by a factor of two require lowpass filters with ω c = π/2, that is, ideal half-band filters. The impulse response of this filter is given by h[n] = ω c π sin(ω c n) ω c n = { 1/2, n = 0, 0, n = ±2, ±4,..., where, for simplicity, we assumed a zero-phase response. In general, any filter with h[n] = h[ n] that satisfies h[n] = 0, for n = ±2, ±4,..., and h[0] = 1/2 is called a half-band filter. Such filters can be shown to satisfy or, on the unit circle, H(z) + H( z) = 1, H(e jω ) + H(e j(ω π) ) = 1. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 19/38

Half-band FIR filters (cont.) H(e jω ) can be shown to have odd symmetric about ω = ±π/2. As a result, we have δ p = δ s and ω p = π ω s. Moreover, M/2 must be an odd number. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 20/38

Equiripple half-band filter design 1 Given the specifications ω s, A p, and A s of the half-band filter, compute δ = min{δ s, δ p }, ω p = π ω s. 2 Design a single band Type II FIR filter G(z) of order M/2 (odd) with ω p = 2ω p, ω s = π, and δ = 2δ using minimax optimization (i.e., equiripple design). Note that since G is Type II, G(e jπ ) = 0. 3 Scale the impulse response g[n] by 1/2, upsample the result by a factor of 2, and set the middle coefficient to 1/2. 4 The result is an impulse response h[n] with system function H(z) given by H(z) = 1 [ ] z M/2 + G(z 2 ), 2 which is a half-band filter with passband cutoff frequency ω p. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 21/38

Filter banks A filter bank is a collection of filters with a common input or a common output. The analysis filter bank splits x[n] into K sub-band signals v k [n], using the analysis filters H k (z). The synthesis filter bank consists of K synthesis filters G k (z) that combine K signals s k [n] into a signal y[n]. If s k [n] = v k [n], for all k, we would like to have y[n] = x[n]. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 22/38

Two-channel filter banks Maximally decimated two-channel filter banks use K = 2. In this case, y[n] can be shown to be given by Y (z) = 1 [T (z)x(z) + A(z)X( z)], 2 where T (z) = H 0 (z)g 0 (z) + H 1 (z)g 1 (z), A(z) = H 0 ( z)g 0 (z) + H 1 ( z)g 1 (z). Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 23/38

Two-channel filter banks (cont.) Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 24/38

Two-channel filter banks (cont.) The system function A(z) (between X( z) and Y (z)) causes aliasing. The system function T (z) (between X(z) and Y (z)) may cause magnitude and phase distortion. That the alias-free filter bank is obtained by enforcing A(z) = 0, in which case we obtain an LTI system with a transfer function T (z)/2. In fact, the filters G 0 (z) (lowpass) and G 1 (z) (highpass) are designed to eliminate the aliasing term. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 25/38

Perfect reconstruction Perfect reconstruction is possible if and only if A(z) = H 0 ( z)g 0 (z) + H 1 ( z)g 1 (z) = 0, T (z) = H 0 (z)g 0 (z) + H 1 (z)g 1 (z) = 2z n D, where n D is an arbitrary integer delay. The condition can also be written as [ ] [ ] [ H0 (z) H 1 (z) G0 (z) 2z n D = H 0 ( z) H 1 ( z) G 1 (z) 0 ]. Given H 0 (z) & H 1 (z), the above system can be solved for G 0 (z) & G 1 (z) as long as m (z) = H 0 (z)h 1 ( z) H 0 ( z)h 1 (z) 0. The resulting synthesis filters guarantee cancellation of aliasing error for any choice of analysis filters. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 26/38

Perfect reconstruction (cont.) In particular, defining G 0 (z) = 2z n D H 1( z) m (z) and G 1 (z) = 2z n D H 0( z) m (z), the condition for perfect reconstruction becomes R(z) + R( z) = 2 where R(z) is a product filter defined as R(z) = z n D H 0 (z)g 0 (z). The product filter R(z) plays a crucial role in analyzing and designing filter banks. It can be shown that R(z) has the form of R(z) = 1 + z 1 R 1 (z 2 ), and, therefore, it must be a half-band filter. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 27/38

Perfect reconstruction (cont.) Thus, to design a two-channel perfect reconstruction filter bank, it is necessary and sufficient to 1 obtain an R(z) = 1 + z 1 R 1(z 2 ), 2 perform the factorization R(z) = z n D H 0(z)G 0(z), 3 assign the synthesis filters G 0(z) and G 1(z). The choice of R(z) and the particular factorization determine the properties of the filter bank. Choosing R(z) = H(z)H(z 1 ) requires the design of a single filter and leads to orthogonal or para-unitary filter banks. Choosing R(z) = H 0 (z)g 0 (z) requires the design of two filters and leads to the more general bi-orthogonal filter banks. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 28/38

Orthogonal FIR filter banks If H 0 (z) and H 1 (z) are FIR filters and m (z) = c 0 z n0, then the synthesis filters G 0 (z) and G 1 (z) will be FIR as well. For example, given an M th -order FIR filter H(z), one can define H 0 (z) = H(z) and H 1 (z) = z M H( z 1 ), which are called conjugate quadrature filters (CQFs). To ensure that the synthesis filters are also FIR we require that or, alternatively, H(z)H(z 1 ) + H( z)h( z 1 ) = 1, H(e jω ) 2 + H(e j(ω π) ) 2 = 1, since H(z) has real coefficients (implying H(e jω ) = H (e jω )). Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 29/38

Orthogonal FIR filter banks (cont.) In this case, m (z) = z M (suggesting c 0 = 1 and n 0 = M). If we also choose n D = M, the synthesis filters are given by G 0 (z) = 2H 1 ( z) = 2z M H( z 1 ), G 1 (z) = 2H 0 ( z) = 2H( z). We note that the product filter is given by in which case R(z) = z n D H 0 (z)g 0 (z) = 2H(z)H(z 1 ), R(z) + R( z) = 2H(z)H(z 1 ) + 2H( z)h( z 1 ) = 2. Thus, the proposed filter bank has the perfect reconstruction property. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 30/38

Orthogonal FIR filter banks (cont.) When the solution of the spectral factorization problem R(z) = = z n D H 0 (z)g 0 (z) = 2H(z)H(z 1 ) exists? It is straightforward to show that H(z) satisfying H(z)H(z 1 )+ +H( z)h( z 1 ) = 1 exists if R(e jω ) 0. Moreover, since R(z) is a half-band filter, we have h[k]h[n + k] = δ[n] k assuming that h[n] is normalized so that h 2 = 1. The above condition suggests that h[n] is orthogonal to its own even translates (with the exception of n = 0), thus leading to the name orthogonal filter banks. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 31/38

Design of a perfect reconstruction CQF bank 1 Design a lowpass zero-phase half-band FIR filter R 0 (z) of order 2M, where M is an odd integer. 2 If δ min = min ω R 0 (e jω ) < 0, replace R 0 (e jω ) by or, alternatively, replace r[n] by R + (e jω ) = R 0 (e jω ) + δ min, r + [n] = r 0 [n] + δ min δ[n]. 3 Scale R + (z) so that R + (e jπ/2 ) = 1/2, viz. R(z) = 1 1 + 2 δ min R +(z). 4 Determine the minimum-phase filter H(z) by solving the spectral factorization problem R(z) = H(z)H(z 1 ). 5 Specify the remaining filters of the bank. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 32/38

Example First, we design an FIR prototype H(z) of order 2M = 14 using [r0,deltamin]=firpm(14,[0 0.425 0.575 1],[1 1 0 0],[1 1]). Note that, in this case, deltamin is negative. We can also obtain various mixed-phase filters. Yet, no linearphase filter is possible. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 33/38

Example (cont.) The figure below shows the magnitude responses of the original half-band filter R 0 (z) and the conjugate quadrature filters H 0 (z) and H 1 (z). Note that R 0 (z) and H 0 (z) have the same stopband edge frequency, but different stopband attenuations. Rule of thumb: to obtain H 0 (z) with A s db, one should start with R 0 (z) having stopband attenuation larger than (2A s + 6) db. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 34/38

Tree-structured filter banks Two-channel filter banks can be used to split a signal into multiple sub-bands. A simple way to design the required filters is by cascading twochannel filter banks. The above is the uniform-band tree structure. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 35/38

Tree-structured filter banks (cont.) Alternatively, we can use the octave-band tree structure. The octave-band tree structure provides a logarithmic decomposition of the spectrum, very useful in coding. This filter bank produces a multiresolution decomposition of x[n], which is closely related to the discrete wavelet transform. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 36/38

Example In many real-life applications, the sub-band signals x 1 [n], x 01 [n], x 001 [n],... tend to be sparse (i.e., dominated by small values). It means that the energy of x[n] is encoded in a relatively small number of relatively large values of its details (i.e., x 1 [n], x 01 [n], x 001 [n],...). Thus, retaining these large values while setting the rest to zero will not have any significant impact on x[n] after reconstruction. As the small-amplitude details are mainly associated with noises and inessential features, the procedure of thresholding has found numerous applications in such areas as 1 signal compression and coding, 2 signal enhancement (de-noising), 3 signal modelling and representation, 4 pattern recognition and machine learning. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 37/38

Example: Image de-noising Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 10 38/38