THE DISCRETE WAVELET TRANSFORM

. 4 THE DISCRETE WAVELET TRANSFORM 4 1

Chaper 4: THE DISCRETE WAVELET TRANSFORM 4 2 4.1 INTRODUCTION TO DISCRETE WAVELET THEORY The bes way o inroduce waveles is hrough heir comparison o Fourier ransforms, a common signal analysis ool. Wavele and Fourier ransforms represen a signal hrough a linear combinaion of heir basis funcions. For Fourier ransforms, he basis funcions are dilaions of cosine and sine signals (each spanning he enire ime inerval). For wavele ransforms, hey are differen ranslaions and dilaions of one funcion ermed he Moher wavele along wih a scaling funcion (each spanning a logarihmically reduced subinerval). The dilaions of boh ses of basis funcions are possible due o heir frequency localizaion, hus allowing us o obain frequency informaion abou he signal being analyzed. This leads o he mos imporan difference beween he wo ses of basis funcions, ime localizaion. The wavele ransform basis funcions are compac, or finie in ime, while he Fourier sine and cosine funcions are no. This feaure allows he wavele ransform o obain ime informaion abou a signal in addiion o frequency informaion. Figure4.1 STFT and DWT Frequency/Time Tiling Fourier ransforms are also capable of obaining ime informaion abou a signal if a windowing procedure is used o creae a Shor Time Fourier ransform (STFT). The window is a square wave which runcaes he sine or cosine funcion o fi a window of a paricular widh. Since he same window is used for all frequencies, he resoluion is he same a all posiions in he ime-frequency plane as seen in figure es. The discree wavele ransform (DWT), on he oher hand, has a window size ha varies frequency scale. This is advanageous for he analysis of signals conaining boh disconinuiies and smooh componens. Shor, high frequency basis funcions are needed for he disconinuiies, while a he same ime, long low frequency ones are needed for he smooh componens. This is exacly he ype of ime-frequency iling you ge from wavele ransforms. Figure 4.1 depics his relaionship by showing how he ime resoluion ges finer as he scale (or 4 2

4 3 4.1 INTRODUCTION TO DISCRETE WAVELET THEORY (a) 0 f f (b) 0 (c) 0 f f (d) 0 (e) 0 Figure4.2 Frequency/Time Tiling of a Discree Signal. (a) Sine Signal wih Disconinuiy (b) Ideniy Transform (c) Discree Fourier Transform (d) Shor Time Fourier Transform (e) Discree Wavele Transform frequency) increases. Each basis funcion is represened by a ile, where he shading corresponds o he value of he expansion coefficien. An example of his represenaion is provided by he analysis of a discree sine signal (wih a disconinuiy) porrayed in Fig. 4.2(a)-(d). The upper wo figures, (b) and (c), display he ideniy and discree Fourier ransform of he given signal. The firs is able o isolae he locaion of he impulse while he second is able o isolae he frequency band of he sine signal, neiher is able o do boh. By using a windowing funcion wih he Fourier ransform (d), one is now able o see boh he impulse and frequency of he sine signal, bu wih a loss of resoluion. The wavele 4 3

Chaper 4: THE DISCRETE WAVELET TRANSFORM 4 4 ransform (e), on he oher hand, is able o achieve beer localizaion of he ime-domain impulse wih a slighly inferior frequency resoluion. For a higher-frequency sine funcion, however, he frequency localizaion would be much worse when using a wavele ransform. From his example, one can see some of he rade-offs beween wavele and Fourier ransforms. An imporan feaure of boh wavele and Fourier ransforms is he orhogonaliy of heir basis funcions, which allows for a unique represenaion of he signal being analyzed. Orhogonaliy enables any funcion, f, o be represened by: f = x < f, x > x (4.1) where x are he basis funcions and < f, x > is he inner produc of f and x. Resuling from his is Parseval s heorem, which relaes he energy of he original signal o ha of he ransformed signal: f 2 = x < f, x > 2 (4.2) The relaion saes ha he energy of he original ime series is preserved in he ransform coefficiens. This means ha he choice of he orhogonal basis you use deermines wheher he energy decomposiion wih respec o he ransformed coefficiens provides ineresing informaion abou he original ime series. Using equaion (4.1), a funcion, f, can be represened by eiher wavele or Fourier basis funcions: Wavele : f () = J,k < f (), ψ(2 J k) >ψ(2 J k) = b 0 φ() + b 2 J +k ψ(2 J k) J,k Fourier : f () = < f (), e jmω 0 > e jmω 0 m (4.3) = m a m e jmω 0 where ψ are he dilaed and ranslaed wavele basis funcions, φ is he scaling funcion (also a wavele basis funcion), and e jmω 0 are he Fourier basis funcions. The resul of he inner produc of f wih he basis funcions is he ransform of ha signal, a, ermed eiher he wavele or Fourier coefficiens. In order o undersand he srucure of he wavele ransform, we can arrange he wavele coefficiens ino he following form: 4 4

4 5 4.1 INTRODUCTION TO DISCRETE WAVELET THEORY f wav () = b 0 φ() + b 1 ψ() [ ] ψ(2) + [b 2 b 3 ] ψ(2 1) ψ(4) + [b 4 b 5 b 6 ψ(4 1) b 7 ] + ψ(4 2) ψ(4 3) (4.4) +... + b (2 j +k)ψ(2 J k) Observe ha he firs wo erms associaed wih he scaling funcion φ() and moher wavele ψ() span he enire ime, e.g., 0 1. The funcions associaed wih he nex wo erms, viz., ψ(2) and ψ(2 1), span 0 1/2 and 1/2 1, respecively, and similarly for subsequen erms wih each spanning smaller and smaller inervals wihou overlap, hus revealing he orhogonaliy of he local basis funcions. Qualiaively speaking, he φ()-erm associaed wih he coefficien b 0 is an averaged value of f () whereas he wavele ψ() associaed wih he coefficien b 1 may be viewed as a differencing operaor spanning he enire range of. Similarly, he nex-level erms [ψ(2) ψ(2 1)] associaed wih [b 2 b 3 ]areagaindifferencingoperaors,eachspanninghalfof he range wihou overlap, and so on. Hence, each addiional level added o he series picks ou he deails lef ou from he series approximaion. Unlike he Fourier basis funcions, here are an infinie number of possible ses of wavele basis funcions. A wavele is formed from a se of filer coefficiens ha mus saisfy a given se of condiions (see he four condiions in page xx of Srang and Nguyen). Any se of filer coefficiens which saisfy he given condiions can be used o creae a wavele funcion. The scaling funcion, or Faher wavele, of order N is hen creaed from a dilaion equaion: N 1 φ() = c k φ(2 k) (4.5) where c k are he filer coefficiens. The wavele funcions can hen be generaed via: k=0 N 1 ψ() = ( 1) k c k φ(2 + k N + 1) (4.6) k=0 whose scaling and he wavele funcions mus also saisfy cerain normalizaion and orhogonalizaion consrains. As menioned previously, here are an infinie number of possible moher waveles. Figure 4.3 shows four of he more common ones: he Haar wavele (also Daubechies order 2 wavele), he 4 5

Chaper 4: THE DISCRETE WAVELET TRANSFORM 4 6 Daubechies order 4 wavele, he Coifle order 3 wavele, and he Symmle order 8 wavele. The order indicaes how smooh he wavele is. I is apparen in Fig. 4.3, ha a higher order, like he Symmle 8 wavele, means a smooher funcion, bu i also means less compacness in ime. The choice and order of he wavele o be used should depend on he dominan feaures of he signal being analyzed. The basis funcions should mach he signal as closely as possible. Figure 4.3 Moher Wavele Basis Funcions To demonsrae he need for an appropriae choice of wavele and illusrae he ime/frequency naure of he wavele ransform, a Doppler signal is analyzed. Figure 4.4 shows he original signal and he wavele ransform using boh he Haar and Symmle 8 waveles. A plo of he wavele coefficiens does no provide much insigh. In order o ge a beer undersanding, he individual frequency levels are separaed ou and he wavele coefficiens ploed as a funcion of ime as shown in Figs. 4.5 and 4.6. Each frequency band spans he enire lengh of ime, bu wih less and less ime resoluion as he frequencies ge lower. On he oher hand, as he frequencies gehe widh of he frequency bands also decrease, meaning ha more frequency resoluion is provided. The magniude of each wavele coefficien is indicaed by he heigh of he line represening i. In he Symmle 8 ransform, he naure of he Doppler signal is shown, he decrease in frequency wih ime is apparen by he sliding of he wavele coefficiens wih each increase in frequency level. The 4 6

4 7 4.2 FAST WAVELET TRANSFORM good represenaion is due o he similariy beween he smoohness of he wavele basis funcions and he Doppler signal. On he opposie exreme, he Haar wavele does no closely mach he Doppler signal a all. This is shown in Fig. 4.6, where he decreasing frequency as a funcion of ime is no nearly as apparen. The Haar ransform does provide a good represenaion of how he ime resoluion increases wih an increase in frequency. Noice ha he spacing beween he wavele coefficiens becomes smaller and smaller as he frequency bands increase. Figure 4.4 Doppler Signal and Is Wavele Transform We can now see clearly an imporan comparison beween he Fourier and he wavele expansions of f (). The Fourier expansion may be advanageous in capuring frequency characerisics in f () whereas he wavele expansion direcly capures he emporal properies of f (). Hence, in a ypical signal processing of vibraion daa, he discree Fourier expansion of f () involves firs he convoluion inegral in he frequency domain, usually via FFT, and hen an inverse FFT. In oher words, in he Fourier expansion he daa mus be ransformed from he ime domain ino he corresponding frequency domain and hen convered again back ino ime domain. On he oher hand, he wavele expansion preserves he emporal naure of he daa while also showing frequency conen during boh he forward and inverse wavele ransforms. 4 7

Chaper 4: THE DISCRETE WAVELET TRANSFORM 4 8-3 S8 Wavele Coefficiens by Level -4-5 -6 Level -7-8 -9-10 -11-12 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Time Figure 4.5 Time/Frequency Represenaion of Doppler Signal via S8 Wavele Transform 4.2 FAST WAVELET TRANSFORM I is generally agreed ha he widespread use of he FFT has been largely due o is compuaional efficiency, ha is, for he sampling size n = 2 J, he FFT has 1 2 n log 2 n muliplicaions. In Fas Wavele Transform, for n daa poin value of f (), viz., f = [ f (0), f (1),..., f (2 J 1)] T (4.7) we would like o obain he wavele coefficiens as expressed in (4.4), he wavele coefficiens b = [b(0), b(1),..., b(2 J 1)] T or is inverse as efficienly as possible. b = Af f = Sb (4.8) To his end, le us consider he Haar funcion which is perhaps he simples wavele basis funcion as shown in Fig. 4.7 for L = 2 J, J = 2. The synhesis marix S can be expressed as 1 1 1 0 1 1 1 0 S = (4.9) 1 1 0 1 1 1 0 1 4 8

4 9 4.2 FAST WAVELET TRANSFORM -3 S8 Wavele Coefficiens by Level -4-5 -6 Level -7-8 -9-10 -11-12 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Time Figure 4.6 Time/Frequency Represenaion of Doppler Signal via Haar Wavele Transform Noe ha he firs column in he above marix, S(1 :4, 1) = [1, 1, 1, 1] T, corresponds o he value of he scaling funcion {φ(), 1 < 1} as is magniude remains for he enire ime inerval. The second column S(1 :4, 2) = [1, 1, 1, 1] T corresponds o he value of he wavele funcion {w(), 1 < 1} as is magniude is 1 for {1 < 1/2} and 1 for {1/2 < 1}. The hird column S(1 :4, 3) = [1, 1, 0, 0] T corresponds o he value of he wavele funcion {w(2), 1 < 1/2} as is magniude is 1 for {1 < 1/4} and 1 for {1/4 < 1/2}. Finally, he fourh column S(1 :4, 4) = [0, 0, 1, 1] T corresponds o he value of he wavele funcion {w(2 1), 1/2 < 1} as is magniude is 1 for {1/2 < 3/4} and 1 for {3/4 < 1}. For compuaional expediency as well as for incorporaing he Parseval heorem, le us inroduce a modified expression for S wih a scale facor (1/ 2) as: r 2 r 2 r 0 r S 4 = 2 r 2 r 0 r 2 r 2 (4.10) 0 r r 2 r 2 0 r which gives uni vecors in he columns and rows since 2r 2 = and 4r 4 = 1. The key aspec ha leads o he Fas Wavele Transform is because, like in he FFT, A can be rearranged as a produc of wo marices: 4 9

Chaper 4: THE DISCRETE WAVELET TRANSFORM 4 10 Figure 4.7 Haar Scaling Funcion and Waveles r r 0 0 1 0 0 0 r r 0 0 r r 0 0 0 0 1 0 r r 0 0 S 4 = 0 0 r r 0 1 0 0 0 0 1 0 0 0 r r 0 0 0 1 0 0 0 1 (4.11) [ ] [ S2 0 S2 0 S 4 = P 0 S 4 2 0 I 2 ] [ ] r r, S 2 = r r For he case of 8 samples, we have 4 10

4 11 4.2 FAST WAVELET TRANSFORM r 3 r 3 r 2 0 r 0 0 0 r 3 r 3 r 2 0 r 0 0 0 r 3 r 3 r 2 0 0 r 0 0 r S 8 = 3 r 3 r 2 0 0 r 0 0 r 3 r 3 0 r 2 0 0 r 0 r 3 r 3 0 r 2 0 0 r 0 r 3 r 3 0 r 2 0 0 0 r r 3 r 3 0 r 2 0 0 0 r which can be decomposed ino wo-marix produc: [ S2 0 S 8 = W 8 0 I 6 ], W 8 = r 2 0 r 2 0 r 0 0 0 r 2 0 r 2 0 r 0 0 0 r 2 0 r 2 0 0 r 0 0 r 2 0 r 2 0 0 r 0 0 0 r 2 0 r 2 0 0 r 0 0 r 2 0 r 2 0 0 r 0 0 r 2 0 r 2 0 0 0 r 0 r 2 0 r 2 0 0 0 r The marix W 8 can be expressed as 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 [ ] 0 0 0 0 1 0 0 0 S4 0 0 0 0 0 1 0 0 W 8 = P S 8, P 8 = 4 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 Using (4.11), S 8 can be decomposed as [ ] [ ] S2 0 S2 0 P 0 S 4 S 8 = 2 0 I 2 [ ] [ ] [ ] S2 0 P 8 S2 0 S2 0 0 I P 0 S 4 6 2 0 I 2 (4.12) (4.13) (4.14) (4.15) For S 16, a similar decomposiion leads o S 16 = [ S8 S 8 ] [ ] S2 0 P 16 0 I 14 (4.16) Now, exensions o an arbirary number of samples, L = 2 J is sraighforward. Finally, i should be poined ha he real uiliy of he above decomposiions is o obain he wavele-ransformed coefficiens b. This can be accomplished by invoking he relaion: 4 11

Chaper 4: THE DISCRETE WAVELET TRANSFORM 4 12 b = Af, A = S T (4.17) I mus be emphasized ha he fas wavele ransform algorihm deailed above for he Haar wavele is applicable o oher waveles. 4.3 MALLAT S PYRAMID ALGORITHM In he preceding secion we have presened he Fas Wavele Transform in erms of linear algebra. A more appealing procedure was presened by S. Malla in 1989 ha uilizes he decomposiion of he wavele ransform in erms of low pass (averaging) filers and high pass (differencing) filers. To illusrae Malla s pyramid algorihm, le s reconsider he case of 8 samples. Since {2J = 8 J = 3}, we need hree filers each for boh low and high pass filers, which for he case of Haar wavele are expressed as Low Pass Filers: L 1 = 1 2 [ 1 1] [ 1 1 ] High Pass Filers: L 2 = 1 2 1 1 L 3 = 1 2 1 1 1 1 1 1 1 1 (4.18) H 1 = 1 2 [ 1 1] ] [ 1 1 H 2 = 1 2 1 1 H 3 = 1 2 1 1 1 1 1 1 1 1 Firs, we group he wavele ransformed coefficien vecor b ino hree ransform levels: b T = [ a 0, b 0, b 1, b 2 ] = [ [b 0 ], [b 1 ], [b 2, b 3 ], [b 4, b 5, b 6, b 7 ] ] (4.19) In he pyramid algorihm, we firs compue he hird-level wavele ransform coefficiens b J 1 = [b 4, b 5, b 6, b 7 ] T via b 2 = H 3 a 3, a 3 = x (4.20) Second, we obain he second-level averaging vecor a J 1 via a 2 = L 3 a 3 (4.21) 4 12

4 13 4.3 MALLAT S PYRAMID ALGORITHM The firs-level wavele ransform coefficiens b 1 is now obained by b 1 = [b 3, b 3 ] T = H 2 a 2 = H 2 L 3 x (4.22) We hen compue he firs-level averaging coefficiens a 0 via a 1 = L 2 a 2 = L 2 L 3 x (4.23) The zeroh-level wavele ransform coefficien b 0 is obained by b 0 = b 1 = H 1 a 1 = H 1 L 2 L 3 x (4.24) Finally, he zeroh-level coefficien ha corresponds o he scaling funcion is obained by a 0 = b 0 = L 1 a 1 = L 1 L 2 L 3 x (4.25) There are waveles whose scaling funcions require more han wo parameers. For example, he Daubechies D 4 wavele scaling and wavele funcions: φ D 4 () = 3 c k φ(2 k) k=0 c 0 = 1 4 (1 + 3), c 1 = 1 4 (3 + 3) c 2 = 1 4 (3 3), c 3 = 1 4 (1 3) ψ D 4 () = 3 h k φ(2 k) k=0 (4.26) h 0 = c 3 = 1 4 (1 3), h 1 = c 2 = 1 4 (3 3) h 2 = c 1 = 1 4 (3 + 3), h 3 = c 0 = 1 4 (1 + 3) 4 13

Chaper 4: THE DISCRETE WAVELET TRANSFORM 4 14 The corresponding low and high pass filers are consrucedd as Low Pass Filers: L 1 = 1 2 c 3 c 2 c 1 c 0 ] 1 2 [ c 3 + c 1 c 2 + c 0 ] (due o wrap around) [ ] [ ] L 2 = 1 c3 c 2 c 1 c 0 1 c3 c 2 c 1 c 0 (wrap around) 2 c 3 c 2 c 1 c 2 0 c 1 c 0 c 3 c 2 c 3 c 2 c 1 c 0 L 3 = 1 c 3 c 2 c 1 c 0 2 c 3 c 2 c 1 c 0 c 1 c 0 c 3 c 2 High Pass Filers: H 1 = 1 2 h 3 h 2 h 1 h 0 ] 1 2 [ h 3 + h 1 h 2 + h 0 ] (due o wrap around) [ ] [ ] h3 h 2 h 1 h 0 h3 h 2 h 1 h 0 (4.27) H 2 = 1 2 H 3 = 1 2 h 3 h 2 h 1 h 0 1 2 h 3 h 2 h 1 h 0 h 3 h 2 h 1 h 0 h 3 h 2 h 1 h 0 h 1 h 0 h 3 h 2 h 1 h 0 h 3 h 2 For he Daubechies wavele, he Malla algorihm deailed in (4.20) - (4.25) is equally applicable. We now formalize Malla s pyramid algorihm for a daa se wih 2 J samples: Decomposiion (o obain he wavele ransform coefficiens b): Iniialize a J = x. For j = J,(J 1),..., 1 compue a j 1 = L j a j and b j 1 = H j a j (4.28) Evenually, afer carrying ou he necessary complex compuaions such as de-noising and convoluions, one would like o reconsruc he desired signal for subsequen signal processing or fuure applicaions. The reconsrucion is simply a ransposiion of he decomposiion, which can be summarized as Reconsrucion (o obain he enhanced signal x): Sar wih a 0 and b 0, b 1,..., b J 1. For j = 1, 2,..., J compue a j = L T j a j 1 + H T j b j 1 ˆx = a J where ˆx is he reconsruced signal (4.29) 4 14

4 15 4.3 MALLAT S PYRAMID ALGORITHM References for Wavele Transforms 1. Graps, Amara, Summer 1995, An Inroducion o Waveles, IEEE Compuaional Sciences and Engineering, 2, n.2, pp 50-61. 2. Srang, G., 1993, Wavele Transforms vs. Fourier Transforms, Bullein (New Series) of AMS, 28(2), pp. 288-305. 3. Malla, S.G., 1989, A heory for muliresoluion signal decomposiion: he wavele represenaion, IEEE Trans. PAMI, 11, 674-693. 4. Williams, John R., Amaraunga, Kevin, 1994, Inroducion o Waveles in Engineering, Inernaional Journal for Numerical Mehods in Engineering, 37, 2365-2388. 5. Daubechies, I., 1992, Ten Lecures on Waveles, SIAM, Philadelphia, PA. 6. Daubechies, I., 1988, Orhonormal bases of compacly suppored waveles, Commun. Pure Appl. Mah., 41, 909-996. 7. Veerli, M. and Kovacevic, J., 1995, Waveles and Subband Coding, Prenice Hall, Englewood Cliffs, NJ. 4 15