Introduction to Orthogonal Transforms. with Applications in Data Processing and Analysis

Transcription

1 i Introduction to Orthogonal Transforms with Applications in Data Processing and Analysis

2 ii

3 Introduction to Orthogonal Transforms with Applications in Data Processing and Analysis October 14, 009 i

4 ii

5 Contents iii cam b ridg e university press Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo Cambridge University Press The Edinburgh Building, Cambridge CB RU, UK Published in the United States of America by Cambridge University Press, New York Information on this title: C Cambridge university Press 007 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 007 Printed in the United Kingdom at the University Press, Cambridge A catalogue record for this publication is available from the British Library Library of Congress Cataloguing in Publication data ISBN-13 ISBN XXXXX-X hardback 0-51-XXXXX-X hardback Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party internet websites referred to in this publication, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.

6 Contents Notation page v 1 Discrete Wavelet Transform 1.1 Multiresolution Analysis scale spaces Wavelet spaces Properties of the scaling and wavelet filters Construction of scaling and wavelet functions Wavelet Expansion Discrete Wavelet Transform (DWT) Iteration algorithm Fast Wavelet Transform (FWT) Filter Bank Implementation of DWT Two-Channel Filter Bank Perfect Reconstruction Filters Two-Dimensional DWT Applications 43 iv

7 Notation General notation iff j = 1 = e jπ/ u + jv Re(u + jv) = u Im(u + jv) = v x n 1 x T A m n A 1 A T A if and only if the imaginary unit complex conjugate of u + jv, equal to u jv real part of u + jv imaginary part of u + jv an n by 1 column vector (bold face lower case letter), the transpose of x, a 1 by n row vector an m by n matrix of m rows and n columns the inverse of matrix A the transpose of matrix A the conjugate transpose of matrix A, i.e., A T = A T = A v

8 vi

9 Notation 1 Provide a platform, a toolbox for a wide variety of data processing and analysis methods based on orthogonal transforms.

10 1 Discrete Wavelet Transform Similar to the Fourier transform and other orthogonal transforms, the continuous wavelet transform (CWT) discussed in the previous chapters also converts a signal, a time function, x(t) into a function in transform domain, in this case a -D function X(s, τ) of arguments s for scale and τ for translation. However, the CWT transform is not an orthogonal transform, as the transform kernel functions ψ s,τ (t) do not form an orthogonal basis to spans the function space containing the signal functions x(t) (Eq.??). In this chapter, we consider the idea of multiresolution analysis or multiresolution approximation (MRA), where a set of orthogonal wavelet functions is constructed to span the function space L (R), just like all the orthogonal transforms discussed before. Specifically, the two parameters s and τ of a wavelet function considered previously ψ s,τ (t) = are discretized in a binary fashion to become: 1 ψ( t τ ) (1.1) s s ψ j,k (t) = 1 ψ(t j n j j ) = j/ ψ( j t k), (j, k Z = {, 1, 0, 1, }) (1.) This function is either an expanded (dilated) version of the mother wavelet ψ(t) if j < 0, or a compressed version of ψ(t) if j > 0. In either case, it is also translated by an integer amount in time to the right if k > 0 or to the left if k < 0. While constructing the specific mother wavelet function ψ(t), we can impose the orthogonality requirement so that all daughter wavelets ψ j,k (t) are orthogonal with respect to not only integer translations (in terms of k), but also binary scaling (in terms of j). In other words, for a given j, these functions form an orthogonal basis that spans a space of scale level j, and these bases across different scale levels are also orthogonal to each other. Moreover, we can further discretize time t, so that the corresponding wavelet transform becomes a discrete wavelet transform (DWT), which is an orthogonal transform that converts a given discrete signal into a set of transform coefficients for the transform kernel functions in terms of both translation k for different temporal locations and j for different scales. After some possible operations

11 Discrete Wavelet Transform 3 Figure 1.1 The nested V j spaces for MRA (e.g., filtering or compression) that can take place in the transform domain, an inverse DWT can be carried out to convert the signal back into time domain. 1.1 Multiresolution Analysis scale spaces Definition 1.1. The multiresolution analysis is based on a sequence of nested scale spaces V j L (R): {0} V V 1 V 0 V 1 V L (R) (1.3) that satisfies the following requirements: Completeness: the union of the nested spaces is the entire function space and their intersection is a set containing 0 as its only member: Self-similarity in scale: Self-similarity in translation and scale: j Z V j = L (R), j Z V j = {0} (1.4) x(t) V 0 iff x( j t) V j, j Z (1.5) x(t) V 0 iff x(t k) V 0, k Z (1.6) Existence of a Riesz basis θk = θ(t k) that spans V 0 : V 0 = span(θ(t k), k Z) (1.7) Note that although Eq.1.6 is only for self-similarity in V 0, i.e., any x(t) V 0 translated by integer k is still in V 0, we can generalize this result to V j. To do so, we replace t by j t in 1.6 and combine it with Eq.1.5 to get: x(t) V 0 iff x( j t) V j iff x( j t k) V j (1.8)

12 4 Chapter 1. Discrete Wavelet Transform If we define a new function y(t) = x( j t), the relationship above can also be expressed as y(t) V j iff y(t j k) V j (1.9) which indicates that any function in V j translated by j k is still in V j. The significance of this set of infinite nested scale spaces V j (j Z) is that any given function x(t) L (R) can be approximated at different levels in each subspace V j L (R). We consider the following two cases. First, consider j > 0, then Vj is spanned by the basis functions φ j,k (t) which are time-compressed (by a factor of j ) versions of φ 0,k (t). As the width of these functions is j times narrower, they have higher resolution and are therefore capable of representing variations of smaller scales for more detailed information in a given signal x(t), than the basis functions φ 0,k (t) in V 0, i.e., V 0 V j. Moreover, if we let j, the scaling function φ j,k (t) has a width 1/ j 0 and height j, i.e., it approaches a Dirac delta function (with certain scaling coefficient). Correspondingly, the space V spanned by such a basis becomes the entire function space L (R) containing all square-integrable functions with details of all levels. This process can be represented as: lim [ j k= x[k]φ j,k (t)] = x(τ)δ(t τ)dτ = x(t) L (R) (1.10) Second, replace j > 0 by j < 0, then V j is spanned by the basis functions φ j,k (t) = φ( j t k) which are j times expanded versions of φ 0,k (t) (with impulse width j times wider), consequently V j spanned by such basis functions contains only the signals of much lower resolution with much less detailed information, i.e., V j V 0. Moreover, if we let j, the scaling function φ j,k (t) has a width 1/ j = j and height j 0, i.e., it approaches a constant 0. Correspondingly the space V spanned by such a basis becomes {0}, a set containing 0 as its only member. Based on the Riesz basis θ k (t), a set of orthonormal functions φ(t), called scaling equation, also father wavelet, can be constructed in frequency domain: Φ(f) = F[φ(t)] = Θ(f) [ k Θ(f k) ] 1/ (1.11) We now show that such a scaling function is orthogonal to itself translated by any integer amount: < φ(t k), φ(t) >= φ(t k)φ(t)dt = δ[k] (1.1) We first represent this orthogonality in frequency domain. As the inner product in the equation is actually the self-correlation of φ(t) evaluated at t = k for all k Z, it can be expressed as the product of the self-correlation r φ (t) and an

13 Discrete Wavelet Transform 5 impulse train with unity interval: r φ (τ) τ =k Z = φ(t τ)φ(t)dt τ =k Z = r φ (τ) k Z δ(τ k) = δ[k] (1.13) The Fourier transform of this product is a convolution of the spectrum of the correlation r φ (t) (Eq.??) and that of the impulse train (Eq.??): Φ(f) k Z δ(f k) = k Z Φ(f k) (1.14) and the Fourier transform of the right-hand side is 1, therefore we have: Φ(f k) = 1 (1.15) k Z Obviously this equation for the orthogonality is satisfied by Φ(f) constructed in Eq.1.11, i.e., the corresponding scaling functions in time domain φ k (t) = φ(t k) form an orthonormal basis that spans V 0. We denote these functions by φ 0,k (t) and write V 0 = span(φ 0,k (t), k Z) (1.16) This result can be generalized to space V j. Replacing t in Eq.1.1 with j t we get: φ( j t k)φ( j t)d( j t) = j φ( j t k) j φ( j t)dt = < φ j,k (t), φ j,0 (t) >= δ[k] (1.17) where we have defined φ j,k (t) = j φ( j t k) = j/ φ( j t k) V j, k Z (1.18) which, according to Eq.1.8, is in V j, i.e., they form an orthogonal basis in V j : V j = span(φ j,k (t), k Z) (1.19) For j > 0, functions φ j,k (t) are compressed in time (shorter duration) but scaled up in value (larger amplitude), and therefore they span a space V j V 0 that can better approximate a given function. The scaling functions in spaces V j of different levels are related. Specifically, the scaling functions φ(t) V 0 V 1 can be expressed in terms of the orthonormal basis φ 1,k (t) = φ(t k) of V 1 : φ(t) = k Z h 0 [k]φ 1,k (t) = k Z h 0 [k]φ(t k) (1.0) where the coefficients h 0 [k] can be found as the projection of φ(t) onto the kth basis function φ 1,k (t) = φ(t k): h 0 [k] =< φ(t), φ(t k) >= φ(t)φ(t k)dt (1.1)

14 6 Chapter 1. Discrete Wavelet Transform Next, this relationship between V 0 and V 1 can be generalized to V j and V j+1. Replacing t in Eq.1.0 with j t l, we get: φ( j t l) = k Z h 0 [k]φ(( j t l) k) = k Z h 0 [k]φ( j+1 t (l + k)) But due to Eq.1.18, we have (1.) φ j,l (t) = k Z h 0 [k]φ j+1,l+k (t) = k Z h 0 [k l]φ j+1,k (t) (1.3) where we have assumed k = l + k, i.e., k = k l. This relationship can also be described in frequency domain. Taking the Fourier transform of Eq.1.0, we get Φ(f) = φ(t)e jπft dt = h 0 [k] φ(t k)e jπft dt k Z = k Z h 0 [k] φ(t )e jπf(t +k)/ d( t ) = 1 h 0 [k]e jkπf φ(t )e jπft / dt k Z = 1 H 0 ( f )Φ(f ) (1.4) where we have assumed t = t k, and H 0 (f) is the discrete-time Fourier transform of the coefficients h 0 [k] for the scaling filter: H 0 (f) = k Z h 0 [k]e jkπf (1.5) Note that for As h 0 [k] is discrete with sampling frequency F = 1 by assumption, H 0 (f ± 1) = H 0 (f) is periodic with a period of F = 1. Eq.1.4 can be further recursively expanded to become: Φ(f) = 1 H 0 ( f )[ 1 H 0 ( f 4 )Φ(f 4 )] = = j=1 1 H 0 ( f j )φ(0) = j=1 1 H 0 ( f j ) (1.6) The last equal sign is based on the assumption that φ(t) is normalized, i.e., its DC component is 1: φ(0) = φ(t)e jπ0t dt = 1 (1.7) The summation index in the discussion above always takes values in the set of integers, e.g., k Z = {,, 1, 0, 1,, }. In the following, for simplicity, we will only indicate the summation index without explicitly specifying the range of values it takes.

15 Discrete Wavelet Transform 7 Example 1.1: Consider a square impulse functions defined as: φ(t) = { 1 0 < t < 1 0 otherwise (1.8) This function is obviously orthonormal to a translated version of itself φ(t k): < φ 0,k (t), φ 0,l >= φ(t k)φ(t l)dt = δ[k], (k, l Z) (1.9) This family of functions can therefore be considered as a set of scaling functions that spans V 0. Any signal x(t) L (R) can be approximated in this space V 0 as: x(t) k c k φ 0,k (t) = k c k φ(t k) (1.30) Replacing t in φ 0,k (t) = φ(t k) by j t and including a normalization factor j/, we can get another set of orthonormal basis functions: φ j,k (t) = j/ φ( j t k), k Z (1.31) As φ(t) = 1 when its argument satisfies 0 < t < 1, φ j,k (t) = φ( j t k) = 1 when its argument satisfies 0 < j t k < 1, i.e., k j < t < 1 j + 1 j (1.3) i.e., φ j,k (t) = φ( j t k) is a square impulse of height j and width 1/ j, and it is shifted k times its width. Obviously these functions are also orthonormal and they span space V j : < φ j,k (t), φ j,l (t) >= δ[k l] (1.33) The basic ideas above are illustrated in Fig.1.. The first four panels show four of the scaling functions φ(t) = φ 0,0 (t) V 0, φ 0,1 (t) = φ(t 1) V 0, φ 1,0 (t) = φ(t) V1, and φ 1,1 (t) = φ(t 1) V 1. Panel 5 shows a given function x(t) V 1 represented as a linear combination of the scaling functions φ 1,k (t): x(t) = 0.5φ 1,0 (t) + φ 1,1 (t) 0.5φ 1,4 (t) (1.34) Finally panel 6 shows that a scaling function φ 0,0 (t) V 0 represented as a linear combination of the basis functions φ 1,k (t) V 1 (Eq.1.3): φ 0,l (t) = h 0 [0]φ 1,l (t) + h 0 [1]φ 1,l+1 (t) = 1 φ 1,k (t) + 1 φ 1,k+1 (t) (1.35) where the coefficients h 0 [0] = h 0 [1] = 1/ are obtained according to Eq.1.1. The ideas illustrated in this example are still valid if the square impulses used as the basis functions in the example are replaced by any family of functions with finite support (with non-zero function values over a finite time duration).

16 8 Chapter 1. Discrete Wavelet Transform Figure 1. Figure 1.3 The nested V j and W j spaces for MRA 1.1. Wavelet spaces Previously we constructed a sequence of nested spaces V j V j+1 in which a given function x(t) L (R) can be approximated at different levels, i.e., the approximation in V j+1 contains more detailed information about the signal than that in V j. In other words, there exist functions (corresponding to the more detailed information) that are in V j+1 but not in V j. The space containing all such functions in V j+1 but not in V j is defined as the wavelet space W j. As W j V j+1 and W j V j = {0}, W j is complementary space of V j V j+1, and we have: V j+1 = W j V j = W j W j 1 V j 1 = (1.36) Expanding this relationship recursively for all V j, and using the completeness property of the scale spaces we get L (R) = j Z W j (1.37) indicating that any function x(t) L (R) can be considered as the sum of infinitely many levels of details. Same as the case that V j is spanned by a set of

17 Discrete Wavelet Transform 9 scaling functions φ j,k (t), as discussed above, here we also assume W j is spanned by a set of basis functions ψ j,k (t), called the wavelet functions, which are orthogonal to themselves, as well as to the scaling functions, i.e., the following should hold for all integer shifts k, l Z at all scale levels j Z: < ψ j,k (t), ψ j,l (t) >= δ[k l] (1.38) < φ j,k (t), ψ j,l (t) >= 0 (1.39) Specifically at scale level j = 0, the orthogonalities above can be written as: < φ(t k), ψ(t) >= < φ(t k), ψ(t) >= φ(t k)ψ(t)dt = δ[k] (1.40) φ(t k)ψ(t)dt = 0 (1.41) Following the same process for the derivation of Eq.1.15 from Eq.1.1 for the orthogonality of the scaling functions, we can also represent these required orthogonalities for the wavelet functions in frequency domain: Ψ(f k) = 1 (1.4) k Φ(f k)ψ(f k) = 0 (1.43) k Consequently, spaces W j and V j spanned respectively by ψ j,k (t) and φ j,l (t) are orthogonal, i.e., W j V j. Moreover, as V j = W j 1 V j 1, it follows that W j W j 1 and W j V j 1 for all j Z. The fact that W j W j 1 also indicates that the wavelet functions ψ j,k (t) are orthogonal with respect to j for different scale levels as well as to k for different integer translations in each scale level. Note that in contrast, the scaling functions φ j,k (t) are not orthogonal across different scale levels. Further more, since all wavelet spaces W j are spanned by ψ j,k (t), the entire function space L (R) = j W j, as the direct sum of these wavelet spaces, is also spanned by these orthogonal wavelet functions: L (R) = span(ψ j,k (t), (j, k Z) (1.44) In the following we will construct the construction of such wavelet functions that satisfy Eq We first consider the case when j = 0 and how corresponding wavelet function ψ(t) = ψ 0,0 (t), called mother wavelet, is related to the scaling functions. Similar to the representation of the father wavelet φ(t) V 1 in Eq.1.0, the mother wavelet ψ(t) V 1 can also be expressed as a linear combination of the basis φ 1,k (t) = φ(t k) of V 1 : ψ(t) = k h 1 [k]φ 1,k (t) = k h 1 [k]φ(t k) (1.45) The coefficients h 1 [k] are obviously different from but certainly related to the coefficients h 0 [k] for φ(t), for the wavelet functions ψ(t) to be orthogonal to the

18 10 Chapter 1. Discrete Wavelet Transform scaling functions φ(t), as to be discussed later. We next replace t by j t l in the equation above to get ψ( j t l) = k h 1 [k]φ(( j t l) k) = k h 1 [k]φ( j+1 t (l + k)) = k h 1 [k l]φ( j+1 t k ) (1.46) But due to Eq.1.18, the above equation becomes: φ( j+1 t k) = (j+1)/ φ j+1,k (t) (1.47) ψ j,l (t) = j/ ψ( j t l) = k h 1 [k l]φ j+1,k (t) (1.48) which defines the wavelet functions ψ j,l (t) that span W j. Eq.1.45 above for the wavelet function can be equivalently represented in frequency domain, similar to Eq.1.4 for the scaling functions: Ψ(f) = F[ψ(t)] = k h 1 [k]f[φ(t k)] = 1 h 1 [k]e jkπ Φ( f ) = 1 H 1 ( f )Φ(f ) (1.49) k where H 1 (f) is the discrete-time Fourier transform for the wavelet filter: H 1 (f) = k h 1 [k]e jkπf (1.50) Note that H 1 (f ± 1) = H 1 (f) is periodic with period 1. Again, same as in Eq.1.6, the wavelet filter can also be recursively expanded to become: Ψ(f) = 1 H 1 ( f ) 1 H 0 ( f j ) (1.51) Also recall that in order to satisfy the admissibility condition (Eq.??), the integral of the wavelet ψ(t) needs to be zero (Eq.??), i.e., its DC component should be zero: Ψ(0) = j= ψ(t)e jπ0t dt = ψ(t)dt = 0 (1.5) For the wavelet functions to be orthonormal and also orthogonal to the scaling functions as required, they obviously need to satisfy certain conditions in terms of the coefficients h 1 [k] or equivalently the wavelet filter H 1 (f). Now we consider the how to construct the wavelet functions that satisfy the required orthogonalities. To do so, we first prove the theorem below, and then construct a wavelet function accordingly.

19 Discrete Wavelet Transform 11 Theorem 1.1. The wavelet functions ψ(t) defined in Eq.1.45 are orthogonal to the scaling functions φ(t k), i.e., Eqs.1.41 and 1.43 hold, if and only if the wavelet filter H 1 (f) is related to the scaling filter H 0 (f) by the following: H 0 (f)h 1 (f) + H 0 (f 1 )H 1(f 1 ) = 0 (1.53) Note that as H i (f ± 1) = H i (f) is periodic, H i (f 1 ) = H i(f + 1 ) for i = 0, 1. Proof: Substituting Eqs.1.4 and 1.49 into Eq.1.43, we get: H 0 ( f k )φ( f k )H 1 ( f k )φ( f k ) k = H 0 ( f k )H 1 ( f k ) φ(f k ) = 0 (1.54) k Separating the even and odd terms in the summation we rewrite the above as: H 0 ( f k )H 1 ( f k ) k φ(f ) k + f (k + 1) f (k + 1) H 0 ( )H 1 ( ) (k + 1) φ(f ) = 0 (1.55) k We replace f/ by f and recall that H 0 (f) and H 1 (f) have period 1 to get H 0 (f )H 1 (f ) k Φ(f k) + H 0 (f 1 )H 1(f 1 ) k Φ(f k 1 ) = 0 (1.56) The proof is complete by realizing that both summations are equal to 1 (Eq.1.15). Next we show that the condition in Eq.1.53 is satisfied by the wavelet filter H 1 (f) constructed below: H 1 (f) = e jπf H 0 (f 1 ) (1.57) Substituting this H 1 (f) into the left-hand side of Eq.1.53 we can easily verify that this equation indeed holds: H 0 (f)e jπf H 0 (f 1 ) H 0(f 1 )ejπ(f+1/) H 0 (f 1) = H 0 (f)e jπf H 0 (f 1 ) + H 0(f 1 )ejπf H 0 (f) = 0 (1.58) The time domain filter coefficients h 1 [n] can be obtained as the inverse Fourier transform of H 1 (f): h 1 [k] = F 1 [ e jπf H 0 (f 1 )] = ( 1)k h 0 [1 k] (1.59)

20 1 Chapter 1. Discrete Wavelet Transform Substituting these coefficients into Eq.1.45, we can construct the wavelet functions that are indeed orthogonal to the scaling functions φ(t k): ψ(t) = k h 1 [k]φ(t k) = k ( 1) k h 0 [1 k]φ(t k) (1.60) Replacing t by j t k, we obtain the wavelet functions ψ j,k (t) = ψ( j t k) that span W j. Theorem 1.. The wavelet function ψ(t) V 0 defined in Eq.1.60 are orthogonal to its integer translations ψ(t l) for all l Z: < ψ(t l), ψ(t) >= Proof: Substituting 1.60 into Eq.1.61, we have < ψ(t l), ψ(t) >= ψ(t l)ψ(t)dt = ( 1) k+k h 0 [1 k]h 0 [1 k ] k k φ(t l)φ(t)dt = δ[l] (1.61) φ((t l) k )φ(t k)dt = ( 1) m+k h 0 [1 k]h 0 [1 m + l] φ(t m)φ(t k)dt k m = ( 1) m+k h 0 [1 k]h 0 [1 m + l]δ[m k] k m = k h 0 [1 k]h 0 [1 k + l] = δ[l] (1.6) Here we have assumed m = l + k and used the fact that φ 1,k (t) are orthonormal (Eq.1.17). The last equal sign is due to a property of the coefficients h 0 [k], to be proven below (Eq.1.68) Example 1.: The scaling function φ(t) considered in the previous example is a square impulse with unit height and width, and the coefficients are h 0 [0] = h 0 [1] = 1/. Now the coefficients for the wavelet functions ψ 1,k (t) can be obtained as and the wavelet function is: ψ(t) = l h 1 [0] = ( 1) 0 h 0 [1 0] = h 0 [0] = 1/ h 1 [1] = ( 1) 1 h 0 [1 1] = h 0 [0] = 1/ h 1 [l] φ[t l] = 1 φ(t) 1 φ(t 1) = (1.63) 1 0 t < 1/ 1 1/ t < 1 0 otherwise (1.64)

21 Discrete Wavelet Transform 13 Figure 1.4 The first two panels of Fig.1.4 show two of the wavelet functions ψ(t) = ψ 0,0 (t) and ψ 0, (t) = ψ(t ) in space W 0. Note that φ 1,k (t) = φ 0,k (t) can be generated by the linear combination of φ 0,k (t) and ψ 0,k (t): φ 1,k (t) = [φ 0,k(t) + ψ 0,k (t)] (1.65) The 3rd panel shows a wavelet function ψ 1,0 (t) = ψ(t) in space W 1. The 4th panel shows a function in space V 0, while the 5th panel shows a function in space W 0. Finally the 6th panel shows a function in space V 1 = V 0 W 0, which can be written as a linear combination of φ 1,k (t), or, equivalently, of φ 0,k (t) and ψ 0,k (t). This example together with the one in previous section illustrate that the Haar transform as discussed in Chapter 6 is actually a wavelet transform. For example, when N =, as shown in Fig.?? and Eq.??, the first row contains the scaling coefficients h 0 [k], the second row contains the wavelet coefficients h 1 [k] Properties of the scaling and wavelet filters We now consider a set of important properties for both the scaling filter h 0 [k] or H 0 (f) and the wavelet filter h 1 [k] or H 1 (f), which will be of great importance for the wavelet filter design to be discussed in future. 1. Normalization: h 0 [k] = h 0 [k + 1] = 1 (1.66) k Z k

22 14 Chapter 1. Discrete Wavelet Transform We integrate both sides of Eq.1.0 with respect to t to get φ(t)dt = h 0 [k] φ(t k)dt = h 0 [k] 1 φ(t )dt k k (1.67) where we have assumed t = t k, i.e.,t = (t k)/. Dividing both sides by φ(t)dt = 0, we get the second equation.. Shift-Orthonormality: The scaling and wavelet filters are orthogonal to themselves translated by any even number of positions: h 0 [k]h 0 [k n] = δ[n] In particular, when n = 0, we have h 0 [k] = 1, k k h 1 [k]h 1 [k n] = δ[n] (1.68) k h 1 [k] = 1 (1.69) Proof: Substituting Eq.1.3 into Eq.1.17 (and replacing k by l), we get δ[l] = < φ j,l (t), φ j,0 (t) >= = k = k k φ j,l (t)φ j,0 (t)dt k h 0 [k l]h 0 [k]δ[k k ] = h 0 [k l]h 0 [k] (1.70) h 0 [k l]h 0 [k ] φ j+1,k (t)φ j+1,k (t)dt k k The proof for k h 1[k] = 1 is identical. 3. Normalization in frequency domain: H 1 (0) = 0, H 0 (0) = (1.71) These can easily obtained by letting f = 0 in Eqs.1.4 and 1.49, and noting φ(0) = 1 (Eq.1.7) and Ψ(0) = 0 (Eq.1.5). Equivalently, we have h 1 [k] = 0, h 0 [k] = (1.7) k which can be also be easily shown by letting f = 0 in Eqs.1.5 and 1.50 and applying the results H 0 (0) = and H 1 (0) = 0 above. 4. Shift-Orthogonalities in frequency domain: H 0 (f) + H 0 (f + 1 ) = H 1 (f) + H 1 (f + 1 ) = k H 0 (f)h 1 (f) + H 0 (f + 1 )H 1(f + 1 ) = 0 (1.73)

23 Discrete Wavelet Transform 15 Proof: Substituting Eq.1.4 into Eq.1.15, we get H 0( f k ) φ( f k ) k = (1.74) We separate the even and odd terms in the summation on the left-hand side to get: H 0( f k ) φ( f k ) + H f (k + 1) 0( ) f (k + 1) φ( ) k k (1.75) But as H 0 (f ± 1) = H 0 (f) is periodic, the above can be written as H 0( f ) φ(f k) + H 0( f + 1 ) φ(f + 1 k) k k = H 0( f ) + H 0( f + 1 ) = 1 (1.76) The last equal sign is due to Eq Replacing f/ by f, we complete the proof. The relation for H 1 (f) can be proven in the same way. The third equation relating H 0 (f) and H 1 (f) is Eq.1.53 already proven above. = Example 1.3: Here we verify that all properties above are satisfied by the scaling and wavelet filters of Haar transform as illustrated in the previous two examples. Recall that the scaling and wavelet coefficients are h 0 [0] = h 0 [1] = 1/ and h 1 [0] = 1/sqrt, h 1 [1] = 1/, with all other h 0 [k] = h 1 [k] = 0 for k 0, 1. We can see immediately that k h 0[k] = h 0 [0] + h 0 [1] = / = and k h 1[k] = h 1 [0] + h 1 [1] = 0. Also, he orthogonality in time domain is obvious. Next we find the DTFT spectra of h 0 [k] and h 1 [k]: H 0 (f) = k h 0 [k]e jkπf = 1 (1 + e jπf ) H 1 (f) = k h 1 [k]e jkπf = 1 (1 e jπf ) We obviously have H 0 (0) = 1/, H 1 (0) = 0 and H 0 (f) + H 0 (f + 1 ) = [1 + cos(πf)] + [1 cos(πf)] = H 1 (f) + H 1 (f + 1 ) = [1 cos(πf)] + [1 + cos(πf)] =

24 16 Chapter 1. Discrete Wavelet Transform H 0 (f)h 1 (f) + H 0 (f + 1 )H 1(f + 1 ) = 0 Moreover, we can find Φ(f) and Ψ(f) of φ(t) and ψ(t) respectively (Eqs.1.8 and 1.64): Φ(f) = Ψ(f) = = φ(t)e jπft dt = ψ(t)e jπft dt = 1 0 1/ 1 jπf (1 e jπf + e jπf ) = and further verify that Eqs.1.4 and 1.49 hold: 1 H 0 ( f )Φ(f ) = 1 1 (1 + e jπf ) and 1 H 1 ( f )Φ(f ) = 1 1 (1 e jπf ) 0 e jπft 1 dt = jπf (1 e jπf ) e jπft dt + 1 1/ 1 jπf (1 e jπf ) = 1 jπf (1 e jπf ) = e jπft dt 1 jπf (1 e jπf ) 1 jπf (1 e jπf ) = Φ(f) 1 jπf (1 e jπf ) = Ψ(f) Construction of scaling and wavelet functions To carry out the wavelet transform of a given signal, the scaling function φ(t) and the wavelet functions ψ(t) need to be specifically determined. In general this is a design process which can be done in one of three different ways: Specify directly φ(t) and ψ(t); Specify their spectra Φ(f) and Ψ(f) in frequency domain; Specify their corresponding filter coefficients h0 [k] and h 1 [k]. Ideally our goal is to find the scaling and wavelet functions with good locality in both time and frequency domains. In the following we will consider these three different methods for the construction the of scaling and wavelet functions, each illustrated by one example. Haar wavelets Based on the discussions above, we can now specifically construct the scaling and wavelet functions following the steps below: 1. Choose the scaling function φ(t) satisfying Eq.1.1: < φ(t k), φ(t) >= δ[k] (1.77) or Φ(f) satisfying Eq.1.15: Φ(f k) = 1 (1.78) k

25 Discrete Wavelet Transform 17 For Haar transform, the scaling function is: φ(t) = { 1 0 t < 1 0 otherwise (1.79). Find scaling coefficients h 0 [k] based on Eq.1.1: h 0 [k] =< φ(t), φ(t k) > (1.80) or H 0 (f) according to Eq.1.4: For Haar transform, we have h 0 [k] = = 1 0 H 0 (f) = Φ(f) Φ(f) φ(t)φ(t k)dt = 1 0 φ(t k)dt φ(t k)dt = 1 { 1 k = 0, 1 0 otherwise 3. Find wavelet coefficients h 1 [k] according to Eq.1.59 (1.81) (1.8) or H 1 (f) according to Eq.1.57 h 1 [k] = ( 1) k h 0 [1 k] (1.83) H 1 (f) = e jπf H 0 (f 1 ) (1.84) For Haar transform, we have: h 1 [k] = ( 1) k h 0 [1 k] = 1 4. Find wavelet function ψ(t) according to Eq k = 0 1 k = 1 0 otherwise (1.85) ψ(t) = k ( 1) k h 0 [1 k]φ(t k) (1.86) or Ψ(f) according to Eq.1.49 Ψ(f) = H 1 ( f )Ψ(f ) (1.87) For Haar transform, we have: 1 0 t < 1/ ψ(t) = h 1 [0]φ 1,0 (t) + h 1 [1]φ 1,1 (t) = 1 / t < 1 0 otherwise (1.88) Based on φ(0) = φ 0,0 (t) and ψ(0) = ψ 0,0 (t), all other ψ j,k (t) can be obtained. Obviously the Haar scaling and wavelet functions φ(t) and ψ(t) have perfect temporal locality. However, similar to the ideal filter discussed before, the drawback of the Haar wavelets is their poor frequency locality, due obviously

26 18 Chapter 1. Discrete Wavelet Transform Figure 1.5 Haar scaling and wavelet functions Figure 1.6 Haar scaling and wavelet functions and their spectra to their sinc-like Φ(f) and Ψ(f) caused by the sharp corners of the rectangular time window in both φ(t) and ψ(t). Meyer wavelets Here we will try to construct a wavelet that has good locality in both time and frequency domains. To do so, we need to avoid sharp discontinuities in

27 Discrete Wavelet Transform 19 both domains. This time we start in frequency domain by considering the spectrum Φ(f) of the scaling function φ(t). We will first define a function for a smooth transition from 0 to 1 and then use it to define a smooth frequency window. While there exist many different functions with a smooth transition between 0 and 1, we here consider a 3rd order polynomial function with a smooth transition between 0 and 1 (Fig.1.7(a)): 0 x < 0 ν(x) = 3x x 3 0 x 1 (1.89) 1 x > 1 The coefficients are chosen so that ν(1/) = 1/ and ν(x) + µ(1 x) = 1, a property needed for the orthogonality requirement Eq.1.15 to be satisfied by the corresponding spectrum Φ(f) defined below: { ν( + 3f) f 0 Φ(f) = (1.90) ν( 3f) f 0 As shown in Fig.1.7(b), Φ (f) = 1 when f 1/3, Φ (f) = 0 when /3 f < 1, and φ (f) + φ (f ± 1) = 1 when 1/3 < f < /3 during the transition interval where the two neighboring copies of Φ(f) overlap, i.e., Eq.1.15 is indeed satisfied. Note that Φ(f) is non-zero only for f < /3 Having obtained Φ(f), we will next find the scaling filter H 0 (f) based on H 0 (f) = Φ(f)/Φ(f) (Eq.1.4). As shown in Fig.1.7(c), Φ(f), a compressed version of Φ(f), is zero for all f except for f < 1/3, therefore Φ(f)/Φ(f) is also zero for all f except when f < 1/3, during which interval Φ(f)/Φ(f) = Φ(f) as Φ(f) = 1. Also, as the scaling filter H 0 (f) is periodic H 0 (f ± 1) = H 0 (f), we can write the scaling filter as: H 0 (f) = k Φ((f k)) = k Φ(f k) (1.91) Given H 0 (f), we can find the wavelet filter H 1 (f) based on Eq.1.57: H 1 (f) = e jπf H 0 (f 1 ) = e jπf k Φ(f k 1) (1.9) and then the spectrum Ψ(f) of the wavelet function ψ(t) based on Eq.1.49: Ψ(f) = 1 H 1 ( f )Φ(f ) = 1 e jπf H 0 ( f 1 )Φ( f ) = 1 e jπf k Φ(f k 1)Φ( f ) (1.93) As is shown in Fig.1.7, Ψ(f) can be written as: 0 f < 1/3 1 Ψ(f) = e jπf Φ(f 1) 1/3 < f < /3 1 e jπf Φ(f/) /3 < f < 4/3 0 f > 4/3 (1.94)

28 0 Chapter 1. Discrete Wavelet Transform Figure 1.7 Construction of Meyer scaling and wavelet functions Finally, the scaling function φ(t) and wavelet function ψ(t) can be obtained by inverse Fourier transform of Φ(f) and Ψ(f), respectively, as shown in Fig.1.8, and the scaling filter coefficients can be found as: h 0 [k] = 1 0 H 0 (f)e jπkf df, k Z (1.95) We see that in the case, there may exist infinite number of coefficients h 0 [k]. Daubechies wavelets Another way to achieve better smoothness in time domain and locality is frequency domain is based on the following observation which is also illustrated in Fig.1.9. If we convolve the highly discontinuous rectangular function x(t) with itself, a smoother triangular function y(t) = x(t) x(t) is obtained. In frequency domain, the spectrum of the rectangular function X(f) = F[x(t)], a sinc function, is raised to its nd power by the convolution to become Y (f) = X (f) with higher frequency components attenuated. If we further convolve this triangular function with itself, a smoother still bell-shaped function z(t) = y(t) y(t) = x(t) x(t) x(t) x(t) is obtained, corresponding to X(f) raised ot its 4th power Z(f) = X 4 (f), with higher frequency compo-

29 Discrete Wavelet Transform 1 Figure 1.8 Meyer scaling and wavelet functions and their spectra Figure 1.9 Getting smoother time function by attenuating higher frequency components nents further suppressed. We see that in general, if the spectrum of a scaling function is raised to a higher power, it becomes better localized in frequency domain and smoother in time domain, due to the attenuation of most of the higher frequency components. Now consider in particular the following identity raised to the 3rd power: 1 = [cos (πf) + sin (πf)] 3 = cos 6 (πf) + 3 cos 4 (πf) sin (πf) + 3 cos (πf) sin 4 (πf) + sin 6 (πf) = cos 6 (πf) + 3 cos 4 (πf) sin (πf) + 3 sin (πf + π/) cos 4 (πf + π/) + cos 6 (πf + π/) The last equal sign is due to these identities: cos(θ) = sin(θ + π/), sin(θ) = cos(θ + π/)

30 Chapter 1. Discrete Wavelet Transform We further define H 0 (f) = [cos 6 (πf) + 3 cos 4 (πf) sin (πf)] (1.96) then we have H 0 (0) = and the above equation becomes: H 0 (f) + H 0 (f + 1 ) = (1.97) As this function H 0 (f) satisfies both the normalization and shift-orthogonality properties of a scaling filter given in Eq.1.71 and 1.73, it can indeed be used as a scaling filter, as the notation suggested. Now all we need is to find H(f) by taking square root of H (f). To do so, we rewrite its expression as: and get: H 0 (f) = cos 4 (πf)[(cos(πf)) + ( 3 sin(πf)) ] = cos 4 (πf) cos (πf) + j 3 sin (πf) (1.98) H 0 (f) = cos (πf)[cos (πf) + j3 sin (πf)] = 1 4 (ejπf + + e jπf )(e jπf + e j πf + 3e jπf 3e jπf ) = 1 ( e j3πf e j πf e 4jπf e j6πf )e j3πf [ 3 ] 3 = h 0 [k]e jkπf e j3πf = h 0 [k]e jkπf (1.99) k=0 k=0 Note that we have dropped the exponential factor e j3πf as the value of H 0 (f) is not changed. Here we have obtained a set of four scaling coefficients for Daubechies 4-point wavelet transform: h 0 [0] = h 0 [] = = 0.683, h 0 [1] = = 1.183, = 0.317, h 0 [3] = = (1.100) The corresponding wavelet coefficients can be obtained according to Eq.1.59 h 1 [k] = ( 1) k h 0 [1 k] as: h 1 [1] = h 0 [0] = 0.683, h 1 [0] = h 0 [1] = 1.183, h 1 [ 1] = h 0 [] = 0.317, h 1 [ ] = h 0 [3] = (1.101) Next, both the scaling function φ(t) and wavelet function ψ(f) can be obtained based on Eqs.1.6 and Alternatively, φ(t) and ψ(t) could also be obtained iteratively by Eqs.1.0 and 1.45 based on an initial Haar scaling function: { 1 0 < t < 1 φ(t) = (1.10) 0 else where

31 Discrete Wavelet Transform 3 Figure 1.10 Iterative approximations of Daubechies scaling and wavelet functions The approximated scaling and wavelet functions from the first six iterations are shown in Fig function daubechies T=3; s=64; t0=1/s; N=T*s; K=4; r3=sqrt(3); % time period in second % sampling rate: s saples/second % sampling period % total number of samples % length of coefficient vector

32 4 Chapter 1. Discrete Wavelet Transform h0=[1+r3 3+r3 3-r3 1-r3]/4; % Daubechies coefficients h1=fliplr(h0); % time reversal of h0 h1(::k)=-h1(::k); % negate odd terms phi=zeros(1,n); % scaling function psi=zeros(1,n); % wavelet function phi0=zeros(1,n); for i=1:s phi0(i)=1; % initialize scaling function for j=1:log(s); for n=1:n phi(n)=0; psi(n)=0; for k=0:3 l=*n-k*s; if (l>0 & l<=n) phi(n)=phi(n)+h0(k+1)*phi0(l); l=*n-k*s; if (l>0 & l<=n) psi(n)=psi(n)+h1(k+1)*phi0(l); phi0=phi; % update scaling function subplot(,1,1) plot(0:t0:t-t0,phi) title( Scaling function ); subplot(,1,) plot(-1:t0:t-1-t0,psi); title( Wavelet function ) 1. Wavelet Expansion As discussed at the beginning of the book in Eq.??, a given signal x(t) L (R) can be approximated as a sequence of square impulse functions of unit height weighted by its discrete samples x k. For simplicity we assume the sampling interval between two signal samples is = 1, and call the square impulse as the scaling function φ(t) V 0, as defined in Eq.1.8, then the signal can be approximated

33 Discrete Wavelet Transform 5 as: x(t) k c[k]φ 0,k φ(t k) (1.103) where c[k] is the kth sample value of the amplitude of the signal. This expression is a linear combination of a set of standard basis functions φ 0,k (k Z) that spans space V 0, i.e., it can be considered as an identity transform. Obviously this approximation can be improved if the more detailed information contained in W 0 is added, i.e., if the signal is approximated in space V 1 = V 0 W 0. Of course the approximation can be further improved in space V = V 0 W 0 W 1 with still more detailed information in W 1 added. In general the approximation can be progressively refined if this process is repeated to include more and more wavelet spaces W j, until j, when x(t) is precisely represented in L (R). In this case, the signal can be written as a linear combination of the orthogonal basis functions φ 0,k (t) and ψ j,k (t) (j, k Z) of L (R): x(t) = c 0,k φ 0,k (t) + d j,k ψ j,k (t) (1.104) k j=0 k where c 0,k is the approximation coefficient: c 0,k =< x(t), φ 0,k (t) >= x(t)φ 0,k (t)dt, (for all k) (1.105) and d j,k is the detail coefficient: d j,k =< x(t), ψ j,k (t) >= x(t)ψ j,k (t)dt, (for all k and j > 0) (1.106) The first term contained in the wavelet expansion of the function x(t) represents the approximation of the function at scale level 0 by the linear combination of the scaling functions φ 0,k (t), and the summation with index j in the second term in the expansion is for the details of different levels contained in the function x(t) approximated by the linear combination of the wavelet functions of progressively higher scales. An Example A continuous function x(t) is defined over the period 0 t < 1 as: { t 0 t < 1 x(t) = (1.107) 0 otherwise We use Haar wavelets, and a starting scale 0. Each individual space (V 0, W 0, W 1, ) is spanned by different number of basis functions. For example, there is only one basis function in spaces V 0 and W 0, while space W 1 is spanned by bases, and space W is spanned by 4 bases c 0 (0) = 1 t φ 0,0 (t)dt = t (t)dt = 1 3 (1.108)

34 6 Chapter 1. Discrete Wavelet Transform Figure 1.11 d 1 (0) = d 0 (0) = t ψ 0,0 (t)dt = t ψ 1,0 (t)dt = t (t)dt t (t)dt = t (t)dt t (t)dt = (1.109) (1.110) d 1 (1) = 1 0 t ψ 1,1 (t)dt = t (t)dt t (t)dt = 3 3 (1.111) Therefore the wavelet series expansion of the function x(t) is x(t) = 1 3 φ 0,0(t) + [ 1 4 ψ 0,0(t)] + [ 3 ψ 1,0(t) 3 3 ψ 1,1(t)] + (1.11) Here the first term is V 0, the second term is W 0, the third term is W 1, and V 1 = V 0 W 0, V = V 1 W 1 = V 0 W 0 W 1 This process can be carried out further. By continuously reducing the scale by half (spaces V 3, V 4, ), higher temporal resolution (always doubled) is achieved. However, at the same time, the frequency resolution is reduced (always halved), as shown in the Heisenberg box. 1.3 Discrete Wavelet Transform (DWT) Iteration algorithm To carry out the discrete wavelet transform, both the signal x(t) and the basis functions φ 0,k (t) and ψ j,k (t) will need to be discretized. The signal becomes a vector x = [x[0],, x[n 1]] T containing a set of N samples taken from a continuous signal x[m] = x(m ), (m = 0, 1,, N 1) (1.113)

35 Discrete Wavelet Transform 7 for some sampling period. Similarly the basis functions φ 0,k (t) and ψ j,k (t) are also discretized to become basis vectors φ 0,k = [, φ 0,k [m], ] T and ψ j,k = [, ψ j,k [m], ] T for all k and all J scale levels j = 0, 1,, J 1. Now the wavelet expansion becomes discrete wavelet transform (DWT) by which the discretized signal x[m] is represented as a weighted sum in the space spanned by the discretized bases φ 0,k and ψ j,k : x[m] = J 1 W φ [0, k]φ 0,k [m] + W ψ [j, k]ψ j,k [m], (m = 0,, N 1) k j=0 k (1.114) This is the inverse wavelet transform where the coefficients or weights are the projections of the signal vector on the orthogonal basis vectors: W φ [0, k] =< x, φ 0,k >= W ψ [j, k] =< x, ψ j,k >= N 1 m=0 N 1 m=0 x[m]φ 0,k [m], (for all k) (1.115) x[m]ψ j,k [m], (for all k and all j > 0) (1.116) where W φ [0, k] and W ψ [j, k] are the approximation coefficient and detail coefficient, respectively. These are the forward wavelet transform. Same as all other orthogonal transforms discussed before, the general application of the discrete wavelet transform is to represent the signal in terms of the DWT coefficients for different scales and translations (similar to the Fourier transform coefficients for different frequencies) in the transform domain, in which various filtering, feature extraction and compression can be carried out. The inverse DWT transform can then be carried out to reconstruct the signal back in time domain. An Example: Assume N = 4-point discrete signal x = [x[0],, x[n 1]] T = [1, 4, 3, 0] T and the discrete Haar scaling and wavelet functions are: The coefficient for V 0 : W φ [0, 0] = 1 3 m= φ 0,0 [m] ψ 0,0 [m] ψ 1,0 [m] ψ 1,1 [m] (1.117) x[m]φ 0,0 [m] = 1 [ ] = 1 (1.118) The coefficient for X 0 : W ψ [0, 0] = 1 3 x[m]ψ 0,0 [m] = 1 [ ( 1) + 0 ( 1)] = 4 m=0 (1.119)

36 8 Chapter 1. Discrete Wavelet Transform The two coefficients for X 1 : W ψ [1, 0] = 1 3 x[m]ψ 1,0 [m] = 1 [1 + 4 ( ) ] = 1.5 m=0 (1.10) W ψ [1, 1] = 1 3 x[m]ψ 1,0 [m] = 1 [ ( )] = 1.5 m=0 In matrix form, we have = (1.11) (1.1) Now the function x[m] (m = 0,, 3) can be expressed as a linear combination of these basis functions: x[m] = 1 [W φ[0, 0]φ 0,0 [m] + W ψ [0, 0]ψ 0,0 [m] + W φ [1, 0]ψ 1,0 [m] + W φ [1, 1]ψ 1,1 [m] ] or in matrix form: = (1.13) (1.14) 1.3. Fast Wavelet Transform (FWT) As shown before, the discrete wavelet transform of a discrete signal x = [x[0],, x[n 1]] T is the process of getting the coefficients: W ψ [j, k] = W φ [0, k] = N 1 m=0 N 1 m=0 x[m]φ 0,k [m] (for all k) (1.15) x[m]ψ j,k [m] (for all k and all j > 0) (1.16) However, as both φ j,l [m] and ψ j,l [m] can be expressed as a linear combination of φ j+1,k [m] as indicated by Eqs.1.3 and 1.48, the two equations above can be

37 Discrete Wavelet Transform 9 written as: and = l = l W φ [j, k] = N 1 m=0 N 1 h 0 [l k] W ψ [j, k] = N 1 m=0 N 1 h 1 [l k] x[m]φ j,k [m] = N 1 m=0 x[m] l m=0 x[m]φ j+1,l (t) = l x[m]ψ j,k [m] = N 1 m=0 x[m] l m=0 x[m]φ j+1,l (t) = l Comparing these equations with a discrete convolution: h 0 [l k]φ j+1,l (t) h 0 [l k]w φ [j + 1, l] (1.17) h 1 [l k]φ j+1,l (t) h 1 [l k]w φ [j + 1, l] (1.18) y[k] = h[k] x[k] = n h[k n]x[n] (1.19) we see that the wavelet transform coefficients W φ [j, k] and W ψ [j, k] at the jth scale can be obtained from the coefficients W φ [j + 1, k] at the (j+1)th scale by: Convolution with time-reversed h0 or h 1 ; Sub-sampling to get every other samples in the convolution. We can therefore write W ψ [j, k] = h 1 [ n] W φ [j + 1, n] n=k W φ [j, k] = h 0 [ n] W φ [j + 1, n] n=k (1.130) Based on these two equations, all wavelet and scaling coefficients W ψ [j, k] and W φ [j, k] for all scale levels of a given signal x can be obtained recursively from the coefficients W φ [J, k] at the highest resolution level (with maximum details), which are the data samples x[m] directly from the signal x(t). As a member of the vector space V J at the highest scale level, the discrete signal can be written as a linear combination of the scaling basis functions φ J,k [m]: x[m] = k W φ [J, k]φ J,k [m], (m = 0,, N 1) (1.131) If we let the kth basis function be a unit impulse at the kth sampling time, i.e., φ J,k [m] = δ[k m] (same as the ith component of a unit vector e j in N-dimensional vector space is e ij = δ[i j]), then the kth coefficient W φ [J, k] is the same as the kth sample of the function x(t). In other words, given W φ [J, k] = x(k), the scaling and wavelet coefficients of the lower scales j < J can be obtained by the subsequent filter bank, as shown on the left-hand side of Fig.1.1. The right-hand side is for signal reconstruction, to be discussed in the following section.

38 30 Chapter 1. Discrete Wavelet Transform Figure 1.1 Filter banks for both forward and inverse DWT The computation cost of the fast wavelet transform (FWT) is the convolutions carried out in each of the filters. The number of data samples in the convolution is halved after each sub-sampling, therefore the total complexity is: O(N + N + N 4 + N ) = O(N) (1.13) i.e., the FWT has linear computational complexity. 1.4 Filter Bank Implementation of DWT As shown before, the forward wavelet transform that converts a given signal vector x into a set of transform coefficients W φ [j, k] and W ψ [j, k] in the transform domain can be implemented by the analysis filter bank. Here we will further show that the inverse wavelet transform for the reconstruction of the signal from the DWT coefficients can be similarly implemented by a synthesis filter bank, as illustrated on the right-hand side of Fig.1.1. In the following we will derive the theory for the design of the the filters G 0 and G 1 in the synthesis filter bank Two-Channel Filter Bank The DWT filter bank shown in Fig.1.1 can be considered as a recursive structure based on a two-channel filter bank, shown in Fig This two-channel filter bank is composed of a low-pass filter h 0 [n] with output a[n] (for approximation) and a high-pass filter h 1 [n] with outputs d[n] (for detail) for the analysis filter bank, and two additional filters g 0 [n] and g 1 [n] for the synthesis filter bank. Our goal is to design the two filters g 0 [n] and g 1 [n] so that their output x is the same as the input x. Once this perfect reconstruction is achieved by the two-channel filter bank at the lowest level, it can also be achieved recursively at all higher levels in the entire filter bank in Fig.1.1.

39 Discrete Wavelet Transform 31 Figure 1.13 Two-channel filter bank According to Eqs.1.17 and 1.18, we have: a[k] = n d[k] = n h 0 [n k]x[n] =< x, h 0 (k) > h 1 [n k]x[n] =< x, h 1 (k) > (1.133) and the output x [n] of the two-channel filter bank is: x [n] = k a[k]g 0 [n k] + k d[k]g 1 [n k], (for all n) (1.134) or in vector form: x = k a[k]g 0 (k) + k d[k]g 1 (k) (1.135) Our goal here is to design the two filters g 0 [n] and g 1 [n] on the right-hand side for the inverse DWT so that the output x [n] = x[n], i.e., the original signal can be perfectly reconstructed after DWT and inverse DWT. For convenience, we will carry out the derivation in the following in frequency domain based on the discrete-time Fourier transforms (DTFT) of signals and the impulse responses of the filters. Note that the DTFT spectra are all periodic with period 1, e.g., H 0 (f + 1) = H 0 (f) (or equivalently H 0 (ω + π) = H 0 (ω)). Based on the down-sampling property of the discrete-time Fourier transform (Eq.??), the outputs of filters H 0 (f) and H 1 (f) can be expressed in frequency domain as: A(f) = 1 [H 0( f )X(f ) + H 0( f + 1 D(f) = 1 [H 1( f )X(f ) + H 1( f + 1 )X( f + 1 )] (1.136) )X( f + 1 )] (1.137) Then, based on the up-sampling property of the DTFT (Eq.??), the overall output x [n] can be expressed as: X (f) = G 0 (f)a(f) + G 1 (f)d(f) = 1 [G 0(f)H 0 (f) + G 1 (f)h 1 (f)] X(f) + 1 [G 0(f)H 0 (f + 1 ) + G 1(f)H 1 (f + 1 )] X(f + 1 ) (1.138)

40 3 Chapter 1. Discrete Wavelet Transform For perfect reconstruction, the output must be identical to the original signal, i.e., X(z) = X (z), we need to have { G0 (f)h 0 (f + 1 ) + G 1(f)H 1 (f + 1 ) = 0 (1.139) G 0 (f)h 0 (f) + G 1 (f)h 1 (f) = These two equations can be written in matrix form as: [ H0 (f + 1 ) H 1(f + 1 ) ] [ ] [ ] G0 (f) G0 (f) = H(f) = H 0 (f) H 1 (f) G 1 (f) G 1 (f) where H is defined as: [ H0 (f + 1 H(f) = ) H 1(f + 1 ) ] H 0 (f) H 1 (f) where (f) is the determinant of H(f): and H 1 (f) = 1 (f) [ ] 0 (1.140) [ H1 (f) H 1 (f + 1 ) ] H 0 (f) H 0 (f + 1 ) (1.141) (f) = H 0 (f + 1 )H 1(f) H 0 (f)h 1 (f 1 ) (1.14) Solving the equation above for G 0 (f) and G 1 (f), we get: [ ] [ ] G0 (f) 0 = H 1 (f) = 1 [ H1 (f) H 1 (f + 1 ) ] [ ] 0 G 1 (f) (f) H 0 (f) H 0 (f + 1 ) = [ H1 (f + 1 ) ] (f) H 0 (f + 1 ) Now G 0 (f) and G 1 (f) can be expressed as: G 0 (f) = (1.143) (f) H 1(f + 1 ), G 1(f) = (f) H 0(f + 1 ) (1.144) Also, if we replace f by f + 1 in H, and notice that H 0(f + 1) = H 0 (f) and H 1 (f + 1) = H 1 (f) have period 1, we get: H(f + 1 [ ] ) = H0 (f) H 1 (f) H 0 (f + 1 ) H 1(f + 1 ) (1.145) and (f + 1 ) = (f). Now we can replace f by f + 1 in the above expression for G 1 (f) to get: G 1 (f + 1 ) = (f) H 0(f) (1.146) Multiplying the two sides of this equation by H 1(f + 1 ) = G 0(f), which is just the first equation in Eq.1.144, we get: G 1 (f + 1 )H 1(f + 1 ) = G 0(f)H 0 (f), i.e. G 1 (f)h 1 (f) = G 0 (f + 1 )H 0(f + 1 ) (1.147)

41 Discrete Wavelet Transform 33 This equation can be substituted back into the two equations in Eq in different ways to get the following four conditions for perfect reconstruction: G 0 (f)h 0 (f) + G 0 (f + 1 )H 0(f + 1 ) = G 1 (f)h 1 (f) + G 1 (f + 1 )H 1(f + 1 ) = G 1 (f)h 0 (f) + G 1 (f + 1 )H 0(f + 1 ) = 0 G 0 (f)h 1 (f) + G 0 (f + 1 )H 1(f + 1 ) = 0 (1.148) Comparing these equations with the three equations in Eq.?? for the properties of H 0 (f) and H 1 (f), we see that these conditions will be satisfied if the following hold: G 0 (f) = H 0 (f), and G 1 (f) = H 1 (f) (1.149) These two relations can be considered as the new conditions for perfect reconstruction, and they will be satisfied if the following is true in time domain for i = 0, 1: gi [n] = h i [ n] is the time reversal of h i [n], so that G i (f) = H i ( f) according to the time reversal property of DTFT (Eq.??); All filter coefficients hi [n] = h i [n] are real, so that H i ( f) = H i (f) according to the time reversal property of DTFT (Eq.??. Now we see that given h 0 [n] and h 1 [n] in the analysis filter bank, g 0 [n] and g 1 [n] in the synthesis filter bank can be easily obtained as shown above for a perfect signal reconstruction. Moreover, based on the DTFT properties of down and up-sampling (Eqs.?? and??), the four biorthogonal relations in Eq are the down and up-sampled versions of G 0 (f)h 0 (f), G 1 (f)h 1 (f), G 1 (f)h 0 (f), G 0 (f)h 1 (f), corresponding to the following four down-sampled convolutions in time domain: g 0 [n] h 0 [n] = k g 1 [n] h 1 [n] = k g 1 [n] h 0 [n] = k g 0 [n] h 1 [n] = k h 0 [k]g 0 [n k] = δ[n] h 1 [k]g 1 [n k] = δ[n] h 1 [k]g 0 [n k] = 0 h 0 [k]g 1 [n k] = 0 (1.150) Comparing the first two convolutions above with the orthonormality property of the scaling filter h 0 and wavelet filter h 1 in Eq.1.68, we also see that here g 0 [n] = h 0 [ n] and g 1 [n] = h 1 [ n] are the time reversal of h 0 [n] and h 1 [n], respectively. If express the four filters h 0, h 1, g 0 and g 1, all shifted by n positions, as four vectors h i (n) = [, h i [k n], ] T and g i (n) = [, h 1 [k n], ] T for i = 0, 1, the

42 34 Chapter 1. Discrete Wavelet Transform four convolutions in Eq can now be written as vector inner products: < g 0 (0), h 0 (n) >= δ[n] < g 1 (0), h 1 (n) >= δ[n] < g 1 (0), h 0 (n) >= 0 These four equations can be further summarized as: < g 0 (0), h 1 (n) >= 0 (1.151) < g i (n), h j (0) >= δ[i j]δ[n], (i, j = 0, 1) (1.15) This is the biorthogonal relationship between the analysis and synthesis filters, as discussed in??. We can now verify that Eq is indeed a perfect reconstruction of the original signal x, if Eq is satisfied. We first assume the given signal x can indeed be expanded in the following form: x = k a k g 0 (k) + k d k g 1 (k) (1.153) where a k and d k are two sets of coefficients which can be found by taking the inner product with h 0 (l) and h 1 (l), respectively, on both sides of this equation: < x, h 0 (l) > = k a k < g 0 (k), h 0 (l) > + k d k < g 1 (k), h 0 (l) > = k a[k]δ[k l] = a l (1.154) and < x, h 1 (l) > = k a k < g 0 (k), h 1 (l) > + k d k < g 1 (k), h 1 (l) > = k d[k]δ[k l] = d l (1.155) We see that the two coefficients a l and d l needed for the expansion of x are exactly the same as a[k] and d[k] in Eq.1.133, used in Eq to generate the output x, i.e., it is indeed the perfect reconstruction of the input signal x. In summary, we can view the two-channel filter bank in Fig.1.13 as a process of signal transform based on two pairs of biorthogonal bases {h 0, g 0 } and {h 1, g 1 }, where g 0 and g 1 are dual to h 0 and h 1, respectively. This transform is essentially the same as what we discussed in??, where a signal x is reconstructed according to Eq.?? as x = k < x, h k > φ k We see that this reconstruction is different from the two-channel filter bank discussed above in that only a pair of dual biorthogonal bases is used, while in the case of the -channel filter bank, two pairs are used.

43 Discrete Wavelet Transform 35 Recall that we discussed a two-point filter bank implementation of Haar transform in??, which is actually a simple example of the general discrete wavelet transform. We list below the Matlab code for the implementation of the two-channel filter bank. Based on this code, the forward discrete wavelet transform for signal decomposition and the inverse DWT for signal reconstruction can be recursively constructed. function y=reconstruction(x,h) N=length(x); % length of signal vector K=length(h); % length of filter (K<N) h=h/norm(h); % normalize h h0=zeros(1,n); h0(1:k)=h; % analysis filter H0 H0=fft(h0); for k=0:n-1 m=mod(k-n/,n)+1; H1(k+1)=-exp(-j**pi*k/N)*conj(H0(m)); % analysis filter H1 G0=conj(H0); G1=conj(H1); % synthesis filters G0 and G1: % Decomposition by analysis filters: A=fft(x); % input signal as initial approximation d=ifft(a.*h1); % filtering of detail d a=ifft(a.*h0); % filtering of approximation a d=d(1::length(d)); % downsampling d a=a(1::length(a)); % downsampling a % Reconstruction by synthesis filters: a=upsample(a,); % upsampling for A d=upsample(d,); % upsampling for D a=ifft(fft(a).*g0); % filtering of a d=ifft(fft(d).*g1); % filtering of d y=a+d; % perfect reconstruction of x As can be seen, here the filtering is carried out in frequency domain by multiplication. Alternatively, the filtering can also be carried out in time domain as a circular convolution, as discussed in??. The code for both forward and inverse DWT transforms is listed below. The input of the forward DWT function includes a vector x for the signal to be transformed and another vector h for the father wavelet coefficients h 0 [k], and the output is a vector w for the DWT coefficients. function w=dwt(x,h) K=length(h); N=length(x); n=log(n); if n~=int16(n)

44 36 Chapter 1. Discrete Wavelet Transform error( Length of data x should be power of ); if K>N error( K should be less than N ); % assume N > K h=h/norm(h); % normalize h h0=zeros(1,n); h0(1:k)=h; H0=fft(h0); for k=0:n-1 m=mod(k-n/,n)+1; H1(k+1)=-exp(-j**pi*k/N)*conj(H0(m)); a=x; n=length(a); w=[]; while n>=k A=fft(a); d=real(ifft(a.*h1)); % convolution d=a*h1 a=real(ifft(a.*h0)); % convolution a=a*h0 d=d(::n); % downsampling d a=a(::n); % downsampling a H0=H0(1::length(H0)); % subsampling H0 H1=H1(1::length(H1)); % subsampling H1 w=[d,w]; % concatenate wavelet coefficients n=n/; w=[a w]; % app signal residual The input of the iverse DWT function include a vector w for the DWT coefficients and a vector h for the father wavelet coefficients h 0 [k], and the output is a vector y for the reconstructed signal x. function y=idwt(w,h) N=length(w); n=log(n); K=length(h); if n~=int16(n) error( Length of data w should be power of ); h0=zeros(1,n); h0(1:k)=h; H0=fft(h0); for k=0:n-1 m=mod(k-n/,n)+1; H1(k+1)=-exp(-j**pi*k/N)*conj(H0(m)); G0=conj(H0); G1=conj(H1); % synthesis filters

45 Discrete Wavelet Transform 37 i=0; while ^i<k i=i+1; % starting scale based on filter length n=^(i-1); a=w(1:n); while n<n d=w(n+1:*n); % get detail a=upsample(a,,1); % upsampling a d=upsample(d,,1); % upsampling d if n==1 a=a ; d=d ; % upsampling 1x1 is column vector n=*n; % signal size is doubled A=fft(a).*G0(1:N/n:N); % convolve a with subsampled G0 D=fft(d).*G1(1:N/n:N); % convolve d with subsampled G1 a=real(ifft(a)); d=real(ifft(d)); a=a+d; y=a; 1.4. Perfect Reconstruction Filters As there are four function variables H 0, H 1, G 0 and G 1 in the two equations in Eq.1.139, there exist multiple designs for the filter banks. Here are three particular ones: Quadrature mirror filters (QMFs) We let H 1 (z) = H 0 ( z), G 0 (z) = H 0 (z), G 1 (z) = H 0 ( z) (1.156) both of the two equations above can be written in terms of H 0 (z). The first equation above becomes: G 0 (z)h 0 ( z) + G 1 (z)h 1 ( z) = H 0 (z)h 0 ( z) H 0 ( z)h 0 (z) = 0 (1.157) and the second equation becomes: G 0 (z)h 0 (z) + G 1 (z)h 1 (z) = H 0 (z)h 0 (z) H 0 ( z)h 0 ( z) = (1.158) where H 0 (z) is so chosen that H0 (z) H 0 ( z) = to satisfy the requirement for perfect reconstruction. Conjugate quadrature filters (CQFs) We let G 0 (z) = H 0 (z 1 ), G 1 (z) = zh 0 ( z), H 1 (z) = z 1 H 0 ( z 1 ) (1.159)

46 38 Chapter 1. Discrete Wavelet Transform and both of the two equations above can be written in terms of H 0. The first equation above becomes: G 0 (z)h 0 ( z) + G 1 (z)h 1 ( z) = H 0 (z 1 )H 0 ( z) zh 0 ( z)z 1 H 0 (z 1 ) = 0 (1.160) and the second equation becomes: G 0 (z)h 0 (z) + G 1 (z)h 1 (z) = H 0 (z 1 )H 0 (z) + zh 0 ( z)z 1 H 0 ( z 1 ) = H 0 (z 1 )H 0 (z) + H 0 ( z)h 0 ( z 1 ) = (1.161) where H 0 (z) is so chosen that the second expression is to satisfy the requirement for perfect reconstruction. Orthonormal (fast wavelet transform) filter We let H 0 (z) = G 0 (z 1 ), H 1 (z) = G 1 (z 1 ), G 1 (z) = z k+1 G 0 ( z 1 ) (1.16) and both of the two equations above can be written in terms of G 0 (z). The first equation above becomes: G 0 (z)h 0 ( z) + G 1 (z)h 1 ( z) = G 0 (z)g 0 ( z 1 ) + G 1 (z)g 1 ( z 1 ) = G 0 (z)g 0 ( z 1 ) z k+1 G 0 ( z 1 )z k 1 G 0 (z) = G 0 (z)g 0 ( z 1 ) G 0 ( z 1 )G 0 (z) = 0 (1.163) and the second equation becomes: G 0 (z)h 0 (z) + G 1 (z)h 1 (z) = G 0 (z)g 0 (z 1 ) + G 1 (z)g 1 (z 1 ) = G 0 (z)g 0 (z 1 ) + [ z k+1 G 0 ( z 1 )][ z k 1 G 0 ( z)] = G 0 (z)g 0 (z 1 ) + G 0 ( z 1 )G 0 ( z) = (1.164) where G 0 (z) is so chosen that the second expression is to satisfy the requirement for perfect reconstruction. Note that P (z) = G 0 (z)g 0 (z 1 ) is the Z- transform of the autocorrelation p[n] = m g 0[m]g 0 [m + n], and the second equation becomes P (z) + P ( z) = Replacing z by z 1/, we get i.e. 1 [P (z) + P ( z)] = 1 (1.165) 1 [P (z1/ ) + P ( z 1/ )] = 1 (1.166) Consider the down sampled version of the function g 0 [n] = g 0[n], and its autocorrelation p [n] = p[n]. As in Z domain we have: P (z) = 1 [P (z1/ ) + P ( z 1/ )] = 1 (1.167)

47 Discrete Wavelet Transform 39 in time domain we have p [n] = m g [m]g [n + m] = m g[m]g[n + m] = δ[n] (1.168) i.e., the down-sampled version of g 0 [n] is orthonormal. Example: There are different ways to design the FIR filter orthonormal impulse response g 0 [n] for the two-channel filter bank. The conditions for perfect construction filters listed above can be inverse Z- transformed to get: H i (z) = G i (z 1 ) h i [n] = g i [ n], (i = 0, 1) (1.169) G 1 (z) = z k+1 G 0 ( z 1 ) g 1 [n] = ( 1) n g 0 [k 1 n] (1.170) i.e., h i is the time-reversed version of g i (i = 0, 1), and g 1 is both time reversed and modulated version of g 0. Once g 0 is determined, the rest can all be determined. 1.5 Two-Dimensional DWT Similar to all orthogonal transforms previously discussed, the discrete wavelet transform can also be applied to two-dimensional signals such as an image. Similar to the 1-D DWT two-channel filter bank shown in Fig.1.13, a -D DWT twochannel filter bank for both analysis and synthesis is shown in Fig.1.14, where the left half is the analysis filter bank for signal decomposition and the right half is the synthesis filter bank for signal reconstruction. The input of the analysis filter bank is a -D signal array treated as the coefficients W φ [j] at scale level j, and its columns are filtered (horizontal filtering) by the low-pass filter H 0 (f) and high-pass filter H 1 (f), and then the columns of the two resulting arrays are further filtered (vertical filtering) by H 0 (f) and H 1 (f) to generate four sets of coefficients at the next scale level j 1, including Wφ h [j 1] low-pass filtered by H 0 (f) in both directions, Wψ h[j 1] high-pass filtered by H 1(f) in horizontal direction, Wψ v[j 1] high-pass filtered in vertical direction, and W ψ d [j 1] highpass filtered in both directions (diagonal). The synthesis filter bank reverse the process to generate a perfectly reconstructed signal as the output. This two-channel filtering can be carried out to the low-pass filtered signal W φ [j 1] to generate four sets of coefficients at the next scale level j, and this process can be further carried out recursively to obtain the complete - D DWT coefficients, as illustrated in Fig Four sets of these coefficients obtained at four consecutive stages of the recursion are shown in Fig Note that the -D DWT coefficients look very much like other -D transforms such as disrete cosine transform and Haar transform, in the sense that the coefficients

48 40 Chapter 1. Discrete Wavelet Transform Figure D two-channel filter bank Figure 1.15 Signal decomposition and reconstruction by -D two-channel filter bank Figure 1.16 Recursion of -D discrete wavelet transform around the top left corner represent low-frequency components of the signal while those around the bottom right corner represent high-frequency components. The Matlab code for both borward and inverse -D DWT transform is listed below. The input of the forward DWT function includes a -D array x for the signal, such as an image, and a vector h for the father wavelet coefficients h 0 [k], and the output is a D array w of the same size as the input array for the DWT coefficients. function w=dwtd(x,h) K=length(h); [M,N]=size(x); if M~=N error( Input should be a square array );

49 Discrete Wavelet Transform 41 Figure D DWT coefficients obtained at four consecutive stages if K>N error( Data size should be larger than size of filter ); n=log(n); if n~=int16(n) error( Length of data x should be power of ); h0=zeros(1,n); h0(1:k)=h; H0=fft(h0); for k=0:n-1 m=mod(k-n/,n)+1; H1(k+1)=-exp(-j**pi*k/N)*conj(H0(m)); a=x; imshow(a,[]); w=zeros(n); n=length(a); while n>=k pause; t=zeros(n,n); for k=1:n % for all n columns