ECE-700 Time-Frequency/Wavelet Notes

Transcription

1 ECE-7 Time-Frequency/Wavele Noes Phil Schnier March, 7 Time-Frequency Analysis and Coninuous Wavele Transform Why Transforms?: In he field of signal processing we frequenly encouner he use of ransforms. Transforms are named such because hey ake a signal and ransform i ino anoher signal, hopefully one which is easier o process or analyze han he original. Essenially, ransforms are used o manipulae signals such ha heir mos imporan characerisics are made plainly eviden. To isolae a signal s imporan characerisics, however, one mus employ a ransform ha is well mached o ha signal. For example, he Fourier ransform, while well mached o cerain classes of signal, does no efficienly exrac informaion abou signals in oher classes. This laer fac moivaes our developmen of he wavele ransform. Limiaions of Fourier Analysis: Le s again consider he Coninuous-Time Fourier Transform (CTFT) pair: X(Ω) = x() = π x()e jω d X(Ω)e jω dω, where we have abbreviaed our earlier noaion B(jΩ) o B(Ω). The Fourier ransform pair supplies us wih our noion of frequency. In oher words, all of our inuiions regarding he relaionship beween he ime domain and he frequency domain can be raced o his paricular ransform pair. I will be useful o view he CTFT as a complex-exponenial signal expansion. Specifically, he inverse CTFT equaion above says ha he ime-domain signal x() can be expressed as a weighed summaion of basis elemens {b Ω (), < Ω < }, where b Ω () := e jω is he basis elemen corresponding o frequency Ω. Since he number of CTFT basis elemens in uncounably infinie, he summaion is accomplished wih an inegral. Noice ha X(Ω), which specifies he conribuion of b Ω () o x(), can be inerpreed as a measure of he similariy beween b Ω () and x(). The Fourier Series (FS) can be considered as a special sub-case of he CTFT ha applies when he ime-domain signal x() is periodic. Recall ha, if x() is periodic wih period T, hen x() can be expressed as a weighed summaion of basis elemens {b k ()} k=, c P. Schnier,

2 where now b k () := e j π T k : x() = X[k] = k= T T T X[k]e j π T k x()e j π T k d. Here, he basis elemens come from a counably infinie se and are parameerized by he discree frequency index k Z. The coefficien X[k], which specifies he conribuion of b k () o x(), can be inerpreed as a measure of similariy beween b k () and x() on he inerval [ T, T ). Though quie popular, Fourier analysis is no always he bes ool o analyze a signal whose characerisics vary wih ime. For example, consider a signal composed of a periodic componen plus a sharp glich a ime, illusraed in ime- and frequency-domains below. x() X(Ω) Ω Ω Ω Fourier analysis is successful in reducing he complicaed-looking periodic componen ino a few simple parameers: he frequencies {Ω,Ω } and heir corresponding magniudes/phases. The glich componen, described compacly in erms of is ime-domain locaion and ampliude, however, is no described efficienly in he frequency domain since i produces a wide spread of frequency componens. Thus, neiher ime- nor frequencydomain represenaions alone give an efficien descripion of he gliched periodic signal: each represenaion disills only cerain aspecs of he signal. As anoher example, consider he linear chirp x() = sin(ω ) illusraed below. x()... Though wrien using he sin( ) funcion, he chirp is no described by a single Fourier frequency. We migh ry o be clever and wrie he ime-varying phase argumen of sin( ) as ϕ() = ϕ() +ϕ(), where ϕ() can be inerpreed as an insananeous frequency. In his case, ϕ() = Ω(), which gives he insananeous frequency Ω() := ϕ() = Ω, which exhibis a simple linear variaion in ime. Bu here we mus be cauious, since his newly defined noion of insananeous frequency is no consisen wih he Fourier noion of frequency. Recall ha he CTFT says ha a signal can be consruced as a superposiion c P. Schnier,

3 of fixed-frequency basis elemens e jω, each spread ou uniformly over all ime. In oher words, here is nohing insananeous abou Fourier frequency. As a hird example, consider a Ω -frequency sinusoid ha is recangularly windowed o exrac only one period. x() X(Ω) Ω Here, our insananeous-frequency argumen migh claim ha { Ω window Ω() = / window, where Ω() akes on exacly wo values. In conras, Fourier heory says ha recangular windowing induces frequency-domain spreading wih a sin(ω) Ω profile, resuling in a coninuum of Fourier frequency componens. Here again, we see ha our noions of insananeous frequency are no compaible wih he sandard Fourier noion of frequency. Time-Frequency Uncerainy Principle: Recall ha Fourier basis b Ω () = e jω exhibis poor ime resoluion a consequence of he fac ha b Ω () for every fixed Ω is uniformly spread over all (, ). By ime resoluion, we mean he abiliy o easily idenify he iming of signal evens like he glich described earlier in an example. A he opposie exreme, a basis composed of shifed Dirac delas b τ () = δ( τ) would have excellen ime resoluion bu errible frequency resoluion, since every Dirac basis elemen is evenly spread over all Fourier frequencies Ω (, ). This can be undersood from he fac ha B τ (Ω) = b τ()e jω d = Ω, for every fixed τ. By frequency localizaion, we mean he abiliy o easily idenify he specral properies of signal componens ha are sinusoidal in naure. These observaions moivae he quesion: Does here exis a basis ha provides simulaneously excellen ime and frequency resoluions? The answer is no really ; here is a fundamenal radeoff beween he ime and frequency resoluions of any basis. This can be undersood from he fac ha no basis elemen can be simulaneously well concenraed in he ime and frequency domains. This idea is made concree below. Consider an arbirary basis elemen, or waveform, b(), whose CTFT will be denoed by B(Ω) = b()e jω. In some cases i will be convenien o express frequency in Herz raher han radians, for which we wrie Ω = πf. We define he emporal and specral ceners as c = E f c = E b() d f B(πf) df, I may be ineresing o noe ha boh E b() and E B(πf) are non-negaive and inegrae o one, hereby saisfying he requiremens of probabiliy densiy funcions. The emporal/specral ceners can hen be inerpreed as he means (i.e., ceners of mass) in he ime/frequency domains, respecively. Ω c P. Schnier, 3

4 and he emporal and specral widhs as = ( c ) E b() d f = (f f c ) E B(πf) df, where E denoes he energy of he waveform, i.e., E = b() d = π B(Ω) dω = B(πf) df via Parseval s heorem. If he waveform is well-localized in ime, hen b() will be concenraed a = c and will be small. If he waveform is well-localized in frequency, hen B(πf) will be concenraed a f = f c and f will be small. If he waveform is well-localized in boh ime and frequency, hen f will be small. The quaniy f is known as he ime-bandwidh produc. From he definiions above one can derive he fundamenal properies below. When inerpreing he properies, i helps o hink of he waveform b() as a prooype ha can be used o generae an enire basis se. For example, he Fourier basis {b Ω (), < Ω < } can be generaed by frequency shifs of b() =, while he Dirac basis {b τ (), < τ < } can be generaed by ime shifs of b() = δ().. and f are invarian o ime and frequency 3 shifs. ( b() ) = ( b( ) ) R f ( B(πf) ) = f ( B(π(f f )) ) f R This implies ha all basis elemens consruced from ime and/or frequency shifs of a prooype waveform b() will inheri he emporal and specral widhs of b().. The ime-bandwidh produc f is invarian o ime-scaling. 4 ( ) ( ) } b(/a) = a b() ( ) ( ) ( ) f b(/a) = a ( ) f b() f b(/a) = f b() a R Observe ha ime-domain expansion (i.e., a > ) increases he emporal widh bu decreases he specral widh, while ime-domain conracion (i.e., a < ) does he opposie. This suggess ha ime-scaling migh be a useful ool for he design of a basis elemen wih a paricular radeoff beween ime and frequency resoluion. On he oher hand, scaling canno simulaneously increase boh ime and frequency resoluion. 3. The ime-bandwidh produc is lower bounded as follows: f 4π This is known as he ime-frequency uncerainy principle. The quaniies and f are analogous o variances. 3 Keep in mind he fac ha b() and B(πf) = R b()e jπf d are alernae descripions of he same waveform; we could have wrien f `b()e jπf in place of f `B(π(f f)) above. 4 The invariance propery holds also for frequency scaling, as implied by he Fourier ransform propery b(/a) a B(πaf). c P. Schnier, 4

5 4. The Gaussian pulse g() achieves he minimum ime-bandwidh produc f = 4π. g() = α π e α, α R G(Ω) = e Ω 4α Noe ha his waveform is neiher bandlimied nor ime-limied, bu well concenraed in boh domains (around he poins c = and f c = ). Properies and can be easily verified using he definiions above. Properies 3 and 4 follow from he Cauchy-Schwarz inequaliy (see he proof below). Since he Gaussian pulse g() achieves he minimum ime-bandwidh produc, i makes for a good prooype waveform, a leas in heory. In oher words, we migh consider consrucing a basis from ime shifed, frequency shifed, ime scaled, or frequency scaled versions of g() o give a range of specral/emporal ceners and specral/emporal resoluions. Since he Gaussian pulse has doubly-infinie ime-suppor, hough, i is no very pracical. Basis consrucion from a prooype waveform is he main concep behind Shor-Time Fourier Analysis and he Coninuous Wavele Transform discussed nex. Proof. Say ha q() is a uni-energy and ime/frequency-cenered version of b(), i.e., q() = Q(πf) = E e jπfc(+c) b( + c ) E e jπfc B(π(f + f c )). so ha q() d = q() d = E E f Q(πf) df = E f Q(πf) df = E ( c ) b() d = ( c ) b() d = (f f c ) B(πf) df = (f f c ) B(πf) df = f. The Cauchy-Schwarz inequaliy says ha x()y()d x() d y() d wih equaliy when x() = ky() for scalar k. Choosing x() = q() and y() = q (), where q () denoes he derivaive of q(), we find ha x() d = y() d = q() d = Y (πf) df = (π) f Q(πf) df = (π) f, c P. Schnier, 5

6 where we used Parseval s equaliy and he fac ha he Fourier ransform of q () equals jω Q(Ω). Thus, he righ side of he Cauchy-Schwarz inequaliy equals (π) f. Since q()q () = q(), we find ha x()y()d = q() d = 4 q () q ()d = 4, assuming ha q() vanishes faser han as ±. Combining wih our earlier resul, we see ha f 4π, wih equaliy when q () = kq() for scalar consan k. This is saisfied by q() of he form q() = α/πe α for α R. Shor-Time Fourier Analysis: We saw earlier ha Fourier analysis is no well suied o describing local changes in frequency conen because he frequency componens defined by he Fourier ransform have infinie (i.e., global) ime suppor. For example, if we have a signal wih periodic componens plus a glich a ime, we migh wan accurae knowledge of boh he periodic componen frequencies and he glich ime. x() X(Ω) Ω Ω Ω The Shor-Time Fourier Transform (STFT) provides a means of join ime-frequency analysis. The STFT pair can be wrien X STFT (Ω,τ) = x() = π x()w( τ)e jω d X STFT (Ω,τ)w( τ)e jω dωdτ assuming real-valued w() for which w() d =. The STFT can be inerpreed as a sliding window CTFT : o calculae X STFT (Ω,τ), slide he cener of window w() o ime τ, window he inpu signal, and compue he CTFT of he resul. This is illusraed in he following figure. The idea is o isolae he signal in he viciniy of ime τ, and hen perform a CTFT analysis in order o esimae he local frequency conen a ime τ. c P. Schnier, 6

7 x() w( τ) x()w( τ) τ Essenially, he STFT uses he basis elemens b Ω,τ () = w( τ)e jω over he range (, ) and Ω (, ). This can be undersood as ime and frequency shifs of he window funcion w(). The STFT basis is ofen illusraed by a iling of he ime-frequency plane, where each ile represens a paricular basis elemen: Ω Ω c Ω c The heigh and widh of a ile represen he specral and emporal widhs of he basis elemen, respecively, and he posiion of a ile represens he specral and emporal ceners of he basis elemen. Noe ha, while he iling diagram above suggess ha he STFT uses a discree se of ime/frequency shifs, he STFT basis is really consruced from a coninuum of ime/frequency shifs. Noe ha we can decrease specral widh Ω a he cos of increased emporal widh by sreching basis waveforms in ime, alhough he ime-bandwidh produc Ω (i.e., he area of each ile) will remain consan. b() a > b(/a) Ω Ω c P. Schnier, 7

8 Our observaions can be summarized as follows: The ime resoluions and frequency resoluions of every STFT basis elemen will equal hose of he window w(). (All STFT iles have he same shape.) The use of a wide window will give good frequency resoluion bu poor ime resoluion, while he use of a narrow window will give good ime resoluion bu poor frequency resoluion. (When iles are sreched in one direcion hey shrink in he oher.) The combined ime-frequency resoluion of he basis, proporional o Ω, is deermined no by window widh bu by window shape. The shape defined by he Gaussian window 5 w() = α/πe α (for α R) gives he highes ime-frequency resoluion, alhough is infinie ime-suppor makes i impossible o implemen. (The Gaussian window resuls in iles wih minimum area.) Coninuous Wavele Transform: The STFT provided a means of (join) ime-frequency analysis wih he propery ha specral/emporal widhs (or resoluions) were he same for all basis elemens. Le s now ake a closer look a he implicaions of uniform resoluion. Consider wo signals composed of sinusoids wih frequency Hz and. Hz, respecively. I may be difficul o disinguish beween hese wo signals in he presence of background noise unless many cycles are observed, implying he need for a many-second observaion. Now consider wo signals wih pure frequencies of Hz and Hz again, a.% difference. Here i should be possible o disinguish he wo signals in an inerval of much less han one second. In oher words, good frequency resoluion requires longer observaion imes as frequency decreases. Thus, i migh be more convenien o consruc a basis whose elemens have larger emporal widhs a low frequencies. The previous example moivaes a muli-resoluion ime-frequency iling of he form: Ω a < a = a > The Coninuous Wavele Transform (CWT) accomplishes he above muli-resoluion iling by ime-shifing and ime-scaling a prooype funcion ψ(), ofen called he moher wavele. The a-scaled and τ-shifed basis elemen is given by [] ψ a,τ () = ( τ ψ a a ) where a,τ R ψ()d = C ψ = 5 The STFT using a Gaussian window is known as he Gabor Transform (946). Ψ(Ω) Ω dω < c P. Schnier, 8

9 The condiions above imply ha Ψ(Ω) is bandpass and ha ψ() is sufficienly smooh. We assume ha ψ() =, so ha he definiion above ensures ψ a,τ () = for all a and τ. In his case, he CWT is defined by he ransform pair X CWT (a,τ) = x() = C ψ x()ψ a,τ()d X CWT (a,τ)ψ a,τ () dτ da a In basic erms, he CWT says ha a waveform can be decomposed ino a collecion of shifed and sreched versions of he moher wavele ψ(). As such, i is usually said ha waveles perform a ime-scale analysis raher han a ime-frequency analysis. The Morle wavele is a classic example of he CWT. I employs a windowed complex exponenial as he moher wavele []: ψ() = π e jω e / Ψ(Ω) = e (Ω+Ω ) / where i is ypical o selec Ω = π log. (See illusraion below.) While his wavele does no exacly saisfy he condiions esablished earlier, i nearly saisfies hem since Ψ() 7 7. Since he correcion is minor, in pracice i is ofen ignored. ψ() Ψ(Ω) imag real While he CWT discussed above is an ineresing heoreical and pedagogical ool, he discree wavele ransform (DWT) is much more pracical. Before shifing our focus o he DWT, we ake a sep back and review some of he basic conceps from he branch of mahemaics known as Hilber Space heory. These conceps will be essenial in our developmen of he DWT. In fac, Hilber Space heory is he mahemaical foundaion for nearly all of signal processing, so i may be ineresing o see wha his foundaion provides for us. c P. Schnier, 9

10 An Inroducion o Hilber Space Theory Hilber spaces provide he mahemaical foundaion for signal processing heory. In his secion we aemp o clearly define some key Hilber space conceps like vecors, norms, inner producs, subspaces, orhogonaliy, orhonormal bases, and projecions. The inen is no o bury you in mahemaics, bu o familiarize you wih he erminology, provide inuiion, and leave you wih a lookup able for fuure reference. Vecor Space: A vecor space consiss of he following four componens:. A se of vecors V,. A field of scalars F (where, for our purposes, F is eiher R or C), 3. The operaion of vecor addiion + (i.e., + : V V V ) 4. The operaion of scalar muliplicaion (i.e, : F V V ) for which he following properies hold. (Assume x,y,z V and α,β F.) a) x + y = y + x (commuaiviy) b) (x + y) + z = x + (y + z) (associaiviy) (αβ) x = α (β x) c) α (x + y) = α x + α y (disribuiviy) (α + β) x = α x + β x d) x V, V s.. x + = x (addiive ideniy) e) x V, ( x) V s.. x + ( x) = (addiive inverse) f) x V, x = x (muliplicaive ideniy) Imporan examples of vecor spaces include i) V = R N, F = R (real N-vecors) ii) V = C N, F = C (complex N-vecors) iii) V = { {x[n]} s.. n= x[n] p < }, F = C (sequences in l p ) iv) V = { f() : f() p d < }, F = C (funcions in L p ) where we have assumed he usual definiions of addiion and muliplicaion. From now on, we will denoe he arbirary vecor space (V, F,+, ) by he shorhand V and assume he usual selecion of (F,+, ). We will also suppress he in scalar muliplicaion, so ha α x becomes αx. A subspace of V is a vecor space defined from he subse M V for which. x,y M, x + y M. x M and α F, αx M (Noe ha every subspace mus conain, and ha V is a subspace of iself.) The span of a se S V is he subspace of V conaining all linear combinaions of vecors in S. When S = {x,...,x N }, span(s) := { N i= α i x i : α i F } c P. Schnier,

11 A subse of linearly-independen vecors {x,...,x N } V is called a basis for V when is span equals V. In such a case, we say ha V has dimension N. We say ha V is infinie-dimensional 6 if i conains an infinie number of linearly independen vecors. V is a direc sum of wo subspaces M and N, wrien V = M N, iff every x V has a unique represenaion x = m + n for m M and n N. (Noe ha his requires M N = {}.) Normed Vecor Space: Now we equip a vecor space V wih a noion of size. A norm is a funcion ( : V R) such ha he following properies hold ( x,y V and α F):. x wih equaliy iff x =,. αx = α x, 3. x + y x + y, (he riangle inequaliy). In simple erms, he norm measures he size of a vecor. Adding he norm operaion o a vecor space yields a normed vecor space. Imporan examples include: i) V = R N, [x,...,x N ] N := i= x i = x x. ii) V = C N, [x,...,x N ] N := i= x i = x H x. ( ii) V = l p, {x[n]} := n= x[n] p) p. iv) V = L p, f() := ( f() p d ) p Inner-Produc Space: Nex we equip a normed vecor space V wih a noion of direcion.. An inner produc is a funcion (, : V V C) such ha he following properies hold ( x,y,z V and α F):. x,y = y,x,. x,αy = α x,y...implying ha αx,y = α x,y, 3. x,y + z = x,y + x,z, 4. x,x wih equaliy iff x =. In simple erms, he inner produc measures he relaive alignmen beween wo vecors. Adding an inner produc operaion o a vecor space yields an inner produc space. Imporan examples include: i) V = R N, x,y := x y. ii) V = C N, x,y := x H y. iii) V = l, {x[n]}, {y[n]} := n= x [n]y[n] iv) V = L, f(),g() := f ()g()d. The inner producs above are he usual choices for hose spaces. The inner produc naurally defines a norm: x := x,x, hough no every norm can be defined from an inner produc. 7 Thus, an inner produc 6 The definiion of an infinie-dimensional basis would be complicaed by issues relaing o he convergence of infinie series. Hence we pospone discussion of infinie-dimensional bases unil he Hilber Space secion. 7 An example for inner produc space L would be any norm f := p q R f() p d such ha p >. c P. Schnier,

12 space can be considered as a normed vecor space wih addiional srucure. Assume, from now on, ha we adop he inner-produc norm when given a choice. The Cauchy-Schwarz inequaliy says x,y x y wih equaliy iff α F s.. x = αy When x,y R, he inner produc can be used o define an angle beween vecors: cos(θ) = x,y x y Vecors x and y are said o be orhogonal, denoed as x y, when x,y =. The Pyhagorean heorem says: x + y = x + y when x y. Vecors x and y are said o be orhonormal when x y and x = y =. x S means x y for all y S. S is an orhogonal se if x y for all x,y S s.. x y. An orhogonal se S is an orhonormal se if x = for all x S. Some examples of orhonormal ses are {[ i) R 3 ] : S =,[ ]}, ii) C N : Subses of columns from uniary marices, iii) l : Subses of shifed Kronecker dela sequences S { {δ[n k]} : k Z }, { } iv) L : S = T p T ( nt) : n Z for T-wide pulse p T () = ( u() u( T) ) and uni sep u(). where in each case we assume he usual inner produc. Hilber Space: Now we consider inner produc spaces wih nice convergence properies ha allow us o define counably-infinie orhonormal bases. A Hilber space is a complee inner produc space. A complee space 8 is one where all Cauchy sequences converge o some vecor wihin he space. For sequence {x n } o be Cauchy, he disance beween is elemens mus evenually become arbirarily small: ǫ >, N ǫ s.. n,m N ǫ, x n x m < ǫ For a sequence {x n } o be convergen o x, he disance beween is elemens and x mus evenually become arbirarily small: ǫ >, N ǫ s.. n N ǫ, x n x < ǫ Examples are lised below (assuming he usual inner producs): i) V = R N, ii) V = C N, 8 The raional numbers provide an example of an incomplee se. We know ha i is possible o consruc a sequence of raional numbers which approximae an irraional number arbirarily closely. I is easy o see ha such a sequence will be Cauchy. However, he sequence will no converge o any raional number, and so he raionals canno be complee. c P. Schnier,

13 iii) V = l iv) V = L (i.e., square summable sequences), (i.e., square inegrable funcions). We will always deal wih separable Hilber spaces, which are hose ha have a counable 9 orhonormal (ON) basis. A counable orhonormal basis for V is a counable orhonormal se S = {x k } such ha every vecor in V can be represened as a linear combinaion of elemens in S: y V, {α k } s.. y = k α k x k. Due o he orhonormaliy of S, he basis coefficiens are given by We can see his via: x k,y = x k, lim n n i= α i x i = lim n α k = x k,y. x k, n i= α i x i = lim n n i= α i x k,x i = α }{{} k δ[k i] (where he second equaliy invokes he coninuiy of he inner produc). In finie n-dimensional spaces (e.g., R n or C n ), any n-elemen ON se consiues an ON basis. In infinie-dimensional spaces, we have he following equivalences concerning an orhonormal se {x,x,x,...} V : a) {x,x,x,... } is an ON basis for V. b) If x i,y = for all i, hen y =. c) y V, y = i xi,y. (Parseval s Equaliy) d) Every y V is a limi of some sequence of vecors in span({x,x,x,... }). Examples of counable ON bases for various Hilber spaces include: i) R N : {e,...,e N } for e i = [,...,,,,...,] wih in he i h posiion, ii) C N : he columns of any uniary N N marix. iii) l : { {δ i [n]} : i Z }, for δ i [n] := δ[n i] (all shifs of he Kronecker sequence) iv) L : o be consruced using he DWT... Say S is a subspace of Hilber space V. The orhogonal complemen of S in V, denoed S, is he subspace defined by he se {x V : x S}. Assuming ha S is closed, we can wrie V = S S. The orhogonal projecion of y ono S, where S is a closed subspace of V, is ŷ = i x i,y x i s.. {x i } is an ON basis for S. Orhogonal projecion yields he bes approximaion of y in S: ŷ = arg min y x. x S 9 A counable se is a se wih a mos a counably-infinie number of elemens. Finie ses are counable, as are any ses whose elemens can be organized ino an infinie lis. Coninuums (e.g., inervals of R) are uncounably infinie. c P. Schnier, 3

14 The approximaion error e := y ŷ obeys he orhogonaliy principle: e S. We illusrae his concep using V = R 3 below bu sress ha he same geomerical inerpreaion applies o any Hilber space. y e ŷ S A proof of he orhogonaliy principle is: e S e,x i = i y ŷ,x i = i y,x i = ŷ,x i i = j x j,y x j,x i i = j x j,y x j,x i i = j y,x j δ j i i = y,x i i c P. Schnier, 4

15 3 Discree Wavele Transform Main Conceps: The discree wavele ransform (DWT) is a represenaion of a signal x() L using an orhonormal basis consising of a counably-infinie se of waveles. Denoing he wavele basis as {ψ k,n () : k,n Z}, he DWT ransform pair is x() = k= n= d k,n = ψ k,n (),x() = d k,n ψ k,n () ψ k,n ()x()d where {d k,n } are he wavele coefficiens. Noe he relaionship o Fourier series and o he sampling heorem: in boh cases we can perfecly describe a coninuous-ime signal x() using a counably-infinie (i.e., discree) se of coefficiens. Specifically, Fourier series enabled us o describe periodic signals using Fourier coefficiens {X[k] : k Z}, while he sampling heorem enabled us o describe bandlimied signals using signal samples {x[n] : n Z}. In boh cases, signals wihin a limied class are represened using a coefficien se wih a single counable index. The DWT can describe any signal in L using a coefficien se parameerized by wo counable indices: {d k,n : k,n Z}. Waveles are orhonormal funcions in L obained by shifing and sreching a moher wavele ψ() L. For example, ψ k,n () = k/ ψ( k n), k,n Z defines a family of waveles {ψ k,n () : k,n Z} relaed by power-of-wo (or dyadic ) sreches. As k increases, he wavele sreches by a facor wo; as n increases, he wavele shifs righ. Noe ha when ψ() =, he normalizaion ensures ha ψ k,n () = for all k, n Z. Power-of-wo sreching is a convenien, hough somewha arbirary, choice. In our reamen of he discree wavele ransform, however, we will focus on his choice. Even wih power-of-wo sreches, here are various possibiliies for ψ(), each giving a differen flavor of DWT. Waveles are consruced so ha {ψ k,n () : n Z} (i.e., he se of all shifed waveles a fixed scale k) describes a paricular level of deail in he signal. As k becomes smaller (i.e., closer o ), he waveles become more fine grained and he level of deail increases. In his way, he DWT can give a muli-resoluion descripion of a signal, very useful in analyzing real-world signals. Essenially, he DWT gives us a discree muli-resoluion descripion of a coninuous-ime signal in L. In he modules ha follow, hese DWT conceps will be developed from scrach using Hilber space principles. To aid he developmen, we make use of he so-called scaling funcion φ() L, which will be used o approximae he signal up o a paricular level of deail. Like wih waveles, a family of scaling funcions can be consruced via shifs and power-of-wo sreches φ k,n () = k/ φ( k n), k,n Z given moher scaling funcion φ(). The relaionships beween waveles and scaling funcions will be elaboraed upon below via heory and example. See [, Ch. 4] for a slighly differen developmen. c P. Schnier, 5

16 Noe ha he inner-produc expression for d k,n above is wrien for he general complexvalued case. In our reamen of he discree wavele ransform, however, we will assume real-valued signals and waveles, for simpliciy. For his reason, we omi he complex conjugaions in he remainder of our DWT discussions. Example: The Haar Sysem: The Haar basis is perhaps he simples example of a DWT basis, and we will frequenly refer o i in our DWT developmen. Keep in mind, however, ha he Haar basis is only an example; here are many oher ways of consrucing a DWT decomposiion. For he Haar case, he moher scaling funcion is defined by φ() = { < else. φ() From he moher scaling funcion, we define a family of shifed and sreched scaling funcions {φ k,n ()} according o φ k,n () = k/ φ( k n), k,n Z, = k/ φ ( k ( n k ) ), φ k,n () k/ n k (n+) k which are illusraed below for various k and n. The second equaion above makes clear he principle ha incremening n by one shifs he pulse one place o he righ. Observe from he figures ha {φ k,n () : n Z} is orhonormal for each k (i.e., along each row of figures). φ, () φ, () φ, () φ, () φ, () φ, () 3 φ, () φ, () φ, () 3 c P. Schnier, 6

17 A Hierarchy of Deail: Given a moher scaling funcion φ() L he choice of which will be discussed laer we consruc scaling funcions a coarseness-level k and shif n as follows: φ k,n () := k/ φ( k n). We hen use V k o denoe he subspace defined by linear combinaions of scaling funcions a he k h level: V k := span{φ k,n () : n Z}. In he Haar sysem, for example, V and V consis of signals wih he characerisics of x () and x () illusraed below, respecively. x () x () We assume ha φ() ensures he following nesing propery is saisfied: V V V V V coarse deailed In oher words, any signal in V k can be consruced as a linear combinaion of more deailed signals in V k. (The Haar sysem gives proof ha a leas one such φ() exiss.) The nesing propery can be depiced using he se-heoreic diagram below, where V is represened by he conens of he larges egg (which includes he smaller wo eggs), V is represened by he conens of he medium-sized egg (which includes he smalles egg), and V is represened by he conens of he smalles egg. V V V Furhermore, we will assume ha lim V k k lim V k k + {φ( n) : n Z} = L (i.e., conains all signals) = {} (i.e, conains only he zero signal) = an orhonormal se. c P. Schnier, 7

18 Essenially, lim k V k = L means ha, for any x() L and any maximally olerable error ǫ >, we can find an approximaion level k such ha x() x k() d < ǫ. Noe ha, while i migh a firs seem ha lim k V k should conain all non-zero consan signals (e.g., x() = a for a R), he only consan signal in L, he space of squareinegrable signals, is he zero signal. Noe ha orhonormal {φ,n () : n Z} implies orhonormal {φ k,n () : n Z} for all k Z. Since {φ k,n () : n Z} is an orhonormal basis for V k, he bes (in L norm) approximaion of x() L a coarseness-level k (i.e., in V k ) is given by he orhogonal projecion x k () = c k,n n= c k,n φ k,n () = φ k,n (),x() x() x k () Vk Haar Approximaion in V k : I is insrucive o consider he approximaion of signal x() L a coarseness-level k of he Haar sysem. For he Haar case, projecion of x() L ono V k is accomplished using he basis coefficiens giving he approximaion c k,n = = (n+) k n k φ k,n ()x()d k/ x()d x k () = = = c k,n φ k,n () n= n= ( ) (n+) k k/ x()d φ k,n () n k (n+) k ( x()d k n= n } k }{{}{{} heigh = k average value of x() in inerval ) k/ φ k,n (). This corresponds o aking he average value of he signal in each inerval of widh k and approximaing he funcion by a consan over ha inerval. (See below.) c P. Schnier, 8

19 x() k x k () The Scaling Equaion: Consider he level- subspace and is orhonormal basis: V = span{φ,n () : n Z} φ,n () = φ( n) Since V V (i.e., V is more deailed han V ) and since φ, () V, here mus exis coefficiens {h[n] : n Z} such ha φ, () = n= φ( ) = φ() = n= n= h[n]φ,n () h[n]φ( n) h[n]φ( n) This is known as he scaling equaion. To be a valid scaling funcion, φ() mus obey he scaling equaion for some coefficien se {h[n]}. The Wavele Scaling Equaion: The difference in deail beween V k and V k will be described using W k, defined as he orhogonal complemen of V k in V k : V k = V k W k. A imes i will be convenien o wrie W k = Vk. This concep is illusraed in he se-heoreic diagram below. V V V W W Suppose now, ha here exiss a moher wavele ψ() L such ha {ψ( n) : n Z} consiues an orhonormal basis for W. Because every V k has an orhonormal basis Unforunaely, proving he exisence of his ψ(), in he case of general φ(), is ouside he scope of his course; he ineresed reader is referred o [, pp ] for hese deails. I should be noed ha, while such ψ() do exis, hey are no unique. c P. Schnier, 9

20 {φ k,n () : n Z} consruced from shifs and sreches of he moher scaling funcion φ(), i is easily shown ha every W k will have an orhonormal basis {ψ k,n () : n Z} whose elemens are consruced from shifs and sreches of he moher wavele: ψ k,n () = k/ ψ( k n). The Haar sysem will soon provide us wih a concree example. Le s focus, for he momen, on he specific case k =. Since W V, here mus exis {g[n] : n Z} such ha ψ, () = ψ( ) = ψ() = n= n= n= g[n]φ,n () g[n]φ( n) g[n]φ( n) This is known as he wavele scaling equaion. To be a valid scaling-funcion/wavele pair, φ() and ψ() mus obey he wavele scaling equaion for some coefficien se {g[n]}. Condiions on {h[n]} and {g[n]}: Here we derive sufficien condiions on he coefficiens {h[n]} and {g[n]} used in he scaling equaion and wavele scaling equaion o ensure, for every k Z, ha he ses {φ k,n () : n Z} and {ψ k,n () : n Z} have he desired orhonormaliy properies previously discussed. We will make repeaed use of our requiremen ha {φ( n) : n Z} is an orhonormal se. For {φ k,n () : n Z} o be orhonormal a all k, we clearly need orhonormaliy when k =. This is equivalen o δ[m] = φ, (),φ,m () = h[n]φ( n), n l δ[m] = = n n= h[n] l h[l]φ( l m) h[l] φ( n),φ( l m) }{{} δ[n (l+m)] h[n]h[n m] Above we used he fac ha φ,m () = φ( m) = φ( ( m)) = φ,( m). So, given orhonormaliy a level k =, we have jus derived a condiion on {h[n]} which is necessary and sufficien for orhonormaliy a level k =. Ye he same condiion is c P. Schnier,

21 necessary and sufficien for orhonormaliy a level k = : δ[m] = φ, (),φ,m () = h[n]φ,n (), n l = n = n= h[n] l h[l]φ,l+m () h[l] φ,n (),φ,l+m () }{{} δ[n (l+m)] h[n]h[n m]. Using inducion, we conclude ha he previous condiion will be necessary and sufficien for orhonormaliy of {φ k,n () : n Z} for all k Z. There is an ineresing frequency-domain inerpreaion of his condiion. If we define p[m] = h[m] h[ m] = n h[n]h[n m] hen we see ha our condiion is equivalen o p[m] = δ[m]. In he z-domain, his yields he pair of condiions P(z) = H(z)H(z ) = P(z / e j π i ) = P(z/ ) + P( z/ ) Puing hese ogeher, i= = H(z / )H(z / ) + H( z / )H( z / ) = H(z)H(z ) + H( z)h( z ) = H(e jω ) + H(e j(π ω) ) where he las propery invokes he fac ha h[n] R and ha real-valued impulse responses yield conjugae-symmeric DTFTs. Thus we find ha {h[n]} is he impulse response of a power-symmeric filer. Recall ha his propery was also shared by he analysis filers in an orhogonal perfec-reconsrucion FIR filerbank. To find condiions on {g[n]} ensuring ha he se {ψ k,n () : n Z} is orhonormal a every k, we can repea he seps above bu wih g[n] replacing h[n], ψ,n () replacing φ,n (), and he wavele-scaling equaion replacing he scaling equaion. This yields δ[m] = n= g[n]g[n m] = G(z)G(z ) + G( z)g( z ) Nex we derive a condiion which guaranees ha W k V k, as required by our definiion W k = V k, for all k Z. Noe ha, for any k Z, W k V k is guaraneed by {ψ k,n : n c P. Schnier,

22 Z} {φ k,n : n Z} which is equivalen o m, = ψ k+, (),φ k+,m () = g[n]φ k,n (), h[l]φ k,l+m () n l = n = n g[n] l g[n]h[n m]. h[l] φ k,n (),φ k,l+m () }{{} δ[n (l+m)] In oher words, a -downsampled version of g[n] h[ n] mus consis only of zeros. This necessary and sufficien condiion can be resaed in he frequency domain as = { G(z)H(z ) } = G(z / e j π p )H(z / e j π p ) p= = G(z / )H(z / ) + G( z / )H( z / ) = G(z)H(z ) + G( z)h( z ). The choice G(z) = ±z P H( z ) for odd P is sufficien (bu perhaps no necessary) o saisfy our condiion, since G(z)H(z ) + G( z)h( z ) = ±z P H( z )H(z ) z P H(z )H( z ) =. In he ime domain, he condiion on G(z) and H(z) can be expressed g[n] = ± ( ) n h[p n] for odd P. Recall ha his propery was saisfied by he analysis filers in an orhogonal perfec reconsrucion FIR filerbank. Finally, noe ha he wo condiions = H(z)H(z ) + H( z)h( z ) and G(z) = ±z P H( z ) for odd P imply = G(z)G(z ) + G( z)g( z ), and hus he laer condiion on G(z) is redundan. To conclude, G(z) = ±z P H( z ) for odd P = H(z)H(z ) + H( z)h( z ) are sufficien o ensure ha boh {φ k,n () : n Z} and {ψ k,n () : n Z} are orhonormal for all k and ha W k V k for all k. Values of {g[n]} and {h[n]} for he Haar Sysem: The scaling equaion was originally wrien as φ, () = h[n]φ,n (). n c P. Schnier,

23 The previous equaion leads o a clever rick: φ,m (),φ, () = φ,m (), n h[n]φ,n () = n = h[m]. h[n] φ,m (),φ,n () }{{} δ[n m] In oher words, we have a way o calculae he coefficiens {h[m]} if we know φ(). In he Haar case h[m] = = = m+ m { φ,m ()φ, ()d φ, ()d m {,} else since φ, () = in he inerval [,) and zero oherwise. Then, choosing P = in g[n] = ( ) n h[p n], we find ha n = g[n] = n = else for he Haar sysem. From he wavele scaling equaion ψ() = n g[n]φ( n) = φ() φ( ) we can see ha he Haar moher wavele and scaling funcion look like: φ() ψ() In he Haar case, i is now easy o see, since V = V W, how ineger shifs of he moher wavele describe he differences beween signals in V and V : c P. Schnier, 3

24 x () V x () V Noe also ha, as k decreases owards, he Haar sysem can be used o describe more and more deailed signals. In fac, any signal in L can be represened arbirarily closely (in erms of he L norm) by choosing k large enough. In he limi k, he Haar sysem can be used o represen any signal in L. Waveles: A Counable Orhonormal Basis for L : Recall ha V k = W k+ V k+ and ha V k+ = W k+ V k+. Puing hese ogeher and exending he idea yields V k = W k+ W k+ V k+ = W k+ W k+ W l V l, l > k + = W k+ W k+ W k+3 = i=k+ W i If we ake he limi as k, we find ha V := lim k V k = i= W i Moreover, from which i follows ha W V and W k V W W k W V and W k 3 V W W k 3.. W k W j k, or, in oher words, all subspaces W k are orhogonal o one anoher. Since he funcions {ψ k,n () : n Z} form an orhonormal basis for W k, he resuls above imply ha {ψ k,n () : n,k Z} consiues an orhonormal basis for V. Given our assumpions abou φ(), we will have V = L, in which case. {ψ k,n () : n,k Z} consiues an orhonormal basis for L c P. Schnier, 4

25 so ha, for any x() L, we can wrie x() = k= n= d k [n] = ψ k,n (),x(). d k [n]ψ k,n () This is he key idea behind he orhogonal DWT sysem ha we have been developing! Relaionship o Filerbanks: Assume ha we sar wih a signal x() L. Denoe he bes approximaion a he h level of coarseness by x (). (Recall ha x () is he orhogonal projecion of x() ono V.) Our goal, for he momen, is o decompose x () ino scaling coefficiens and wavele coefficiens a higher levels. Since x () V and V = V W, here exis coefficiens {c [n]}, {c [n]}, and {d [n]} such ha x () = n = n c [n]φ,n () c [n]φ,n () + n d [n]ψ,n (). Using he fac ha {φ,n () : n Z} is an orhonormal basis for V, in conjuncion wih he scaling equaion, c [n] = x (),φ,n () = c [m]φ,m (),φ,n () = m = m m c [m] φ,m (),φ,n () c [m] φ( m), l h[l]φ( l n) = m = m c [m] l c [m]h[m n]. h[l] φ( m),φ( l n) }{{} δ[m l n] The previous expression indicaes ha {c [n]} resuls from convolving {c [m]} wih a ime-reversed version of h[m] hen downsampling by facor wo. c [m] H(z ) c [n] Using he fac ha {ψ,n () : n Z} is an orhonormal basis for W, in conjuncion wih c P. Schnier, 5

26 he wavele scaling equaion, d [n] = x (),ψ,n () = c [m]φ,m (),ψ,n () m = c [m] φ,m (),ψ,n () m = c [m] φ( m), g[l]φ( l n) m l = m = m c [m] l c [m]g[m n]. g[l] φ( m),φ( l n) }{{} δ[m l n] The previous expression indicaes ha {d [n]} resuls from convolving {c [m]} wih a ime-reversed version of g[m] hen downsampling by facor wo. c [m] G(z ) d [n] Puing hese wo operaions ogeher, we arrive a wha looks like he analysis porion of an FIR filerbank: H(z ) c [n] c [m] G(z ) d [n] We can repea his process a he nex higher level of coarseness. Since V = W V, here exis coefficiens {c [n]} and {d [n]} such ha x () = n = n c [n]φ,n () d [n]ψ,n () + n c [n]φ,n () Using he same seps as before, we find ha c [n] = m d [n] = m c [m]h[m n] c [m]g[m n] which gives a cascaded analysis filerbank: c P. Schnier, 6

27 c [m] H(z ) c [n] H(z ) G(z ) d [n] c [l] G(z ) d [m] If we use V = W W W 3 W k V k o repea his process up o he k h level, we ge he ieraed analysis filerbank below. c k [q] H(z ) c k [s] c [n] H(z )... G(z ) d k [r]. c [m] H(z ) G(z ) d 3 [p] H(z ) G(z ) d [n] c [l] G(z ) d [m] As we migh expec, signal reconsrucion can be accomplished using cascaded wo-channel synhesis filerbanks. Using he same assumpions as before, we have: c [m] = x (),φ,m () = c [n]φ,n () + n = n = n n c [n] φ,n (),φ,m () + }{{} n h[m n] c [n]h[m n] + n d [n]ψ,n (),φ,m () d [n] ψ,n (),φ,m () }{{} g[m n] d [n]g[m n] which can be implemened using he block diagram below. c [n] H(z) + c [m] d [n] G(z) The same procedure can be used o derive c [m] = n c [n]h[m n] + n d [n]g[m n], from which we ge he diagram: c P. Schnier, 7

28 c [n] H(z) c [m] d [n] G(z) + H(z) d [m] G(z) + c [l] To reconsruc from he k h level, we can use he ieraed synhesis filerbank: c k [s] H(z) c k [q] d k [r] G(z) H(z) c [n] d 3 [p] G(z) + H(z) c [m] d [n] G(z) + H(z) d [m] G(z) + c [l] The able below makes a direc comparison beween waveles and he wo-channel orhogonal PR-FIR filerbanks. Discree Wavele Transform -Channel Orhogonal PR-FIR Filerbank Analysis LPF: H(z ) H (z) Power Symmery: H(z)H(z ) + H( z)h( z ) = H (z)h (z ) + H ( z)h ( z ) = Analysis HPF: G(z ) H (z) Specral Reverse: G(z) = ±z P H( z ), P odd H (z) = ±z (N ) H ( z ), N even Synhesis LPF: H(z) G (z) = z (N ) H (z ) Synhesis HPF: G(z) G (z) = z (N ) H (z ) From he able above, we see ha he discree wavele ransform ha we have been developing is idenical o wo-channel orhogonal PR-FIR filerbanks in all bu a couple deails.. Orhogonal PR-FIR filerbanks employ synhesis filers wih wice he gain of he analysis filers, whereas in he DWT he gains are equal.. Orhogonal PR-FIR filerbanks employ causal filers of lengh N, whereas he DWT filers are no consrained o be causal. For convenience, however, he wavele filers H(z) and G(z) are usually chosen o be causal. For boh o have even impulse response lengh N, we require ha P = N. Iniializaion of he Wavele Transform: The filerbanks developed in he las secion sar wih he signal represenaion {c [n] : n Z} and break he represenaion down ino wavele coefficiens and scaling coefficiens a lower resoluions (i.e., higher levels k). Of course, we could easily sar a a non-zero level K and break he represenaion down from c P. Schnier, 8

29 here; K = jus leads o simple noaion. The quesion remains, hough: how do we ge he iniial coefficiens {c [n]}? From heir definiion, we see ha he scaling coefficiens can be wrien using a convoluion: c [n] = φ( n),x() = φ( n)x()d = φ( ) x() =n, which suggess ha he proper iniializaion of he wavele ransform is accomplished by passing he coninuous-ime inpu x() hrough an analog filer wih impulse response φ( ) and sampling is oupu a ineger imes. x() φ( ) = m c [m] H(z ) G(z ) c [n] d [n] Pracically speaking, however, i is very difficul o build an analog filer wih impulse response φ( ) for ypical choices of scaling funcion. The mos ofen-used approximaion is o se c [n] = x[n]. This iniializaion would be exac if x() was bandlimied o Hz and CTFT Φ(Ω) = for Ω ( π,π] rad/s, bu i is no exac in general. (A similar propery could be saed if we sared he DWT a a non-zero level K.) However, if we regard he wavele ransform primarily as a discree-ime muli-resoluion analysis/synhesis ool, hen he iniializaion (or coninuous-o-discree conversion) sage is no so imporan. Regulariy Condiions, Compac Suppor, and Daubechies Waveles: Up unil now we have been somewha vague abou he requiremens for he condiions V = L and φ( n),φ() = δ[n]. As we shall soon see, he condiions on {h[n]} and {g[n]} developed hus far are no enough o yield φ() saisfying hese condiions! In his secion, herefore, we will ake a closer look a wha happens when k and arrive a a popular family of filer coefficiens h[n] and g[n] which give coninuous φ() and ψ() ha saisfy our desired wavele sysem properies. These are known as Daubechies Waveles. Recall he ieraed synhesis filerbank. Applying he Noble ideniies, we can move he up-samplers before he filers, as illusraed below. c i [q] i i k= H(zk ) d i [q] i G(z i ) i k= H(zk ) c [l] d 3 [p] 8 G(z 4 )H(z )H(z) d [n] 4 G(z )H(z) d [m] G(z) c P. Schnier, 9

30 The properies of he i-sage cascaded lowpass filer H (i) (z) = i k= H(z k ), i in he limi i give an imporan characerizaion of he wavele sysem. Bu how do we know ha lim i H (i) (e jω ) converges o a response in L? In fac, here are some raher sric condiions on H(e jω ) ha mus be saisfied for his convergence o occur. Wihou such convergence, we migh have a finie-sage perfec reconsrucion filerbank, bu we will no have a counable wavele basis for L. Below we presen some regulariy condiions on H(e jω ) ha ensure convergence of he ieraed synhesis lowpass filer. (Noe ha convergence of he lowpass filer implies convergence of all oher filers in he bank.) Le us denoe he impulse response of H (i) (z) by h (i) [n]. Wriing in he ime domain, we have H (i) (z) = H(z i )H (i ) (z) h (i) [n] = k h[k]h (i ) [n i k]. Now define he funcion φ (i) () = i/ n h (i) [n]p [ n i, n+ i )(), where p [a,b) () denoes he heigh- pulse wih suppor on he inerval [a,b): The definiion of φ (i) () implies p [a,b) () = { [a,b) / [a,b) h (i) [n] = i/ φ (i) (), [ n i, n+ i ) h (i ) [n i k] = (i )/ φ (i ) ( k), [ n i, n+ i ) and plugging he wo previous expressions ino he equaion for h (i) [n] yields φ (i) () = k h[k]φ (i ) ( k). Thus, if φ (i) () converges poinwise o a coninuous funcion, hen he limiing funcion mus saisfy he scaling equaion, so ha lim i φ (i) () = φ(). Daubechies [] showed ha, for poinwise convergence of φ (i) () o a coninuous funcion in L, i is sufficien ha H(e jω ) can be facored as H(e jω ) = ( ) + e jω P R(e jω ), P, c P. Schnier, 3

31 for R(e jω ) such ha sup R(e jω ) < P. ω Here P denoes he number of zeros ha H(e jω ) has a ω = π. Such condiions are called regulariy condiions because hey ensure he regulariy, or smoohness, of φ(). In fac, if we make he previous condiion sronger: sup R(e jω ) < P l, l, ω hen lim i φ (i) () = φ() for φ() ha is l-imes coninuously differeniable. There is an ineresing and imporan by-produc of he preceding analysis. If h[n] is a causal lengh-n filer, i can be shown ha h (i) [n] is causal wih lengh N (i) = ( i )(N )+. By consrucion, hen, φ (i) () will be zero ouside he inerval [, (i )(N )+ ). i Assuming ha he regulariy condiions are saisfied so ha lim i φ (i) () = φ(), i follows ha φ() mus be zero ouside he inerval [,N ]. In his case we say ha φ() has compac suppor. Finally, he wavele scaling equaion implies ha, when φ() is compacly suppored on [,N ] and g[n] is lengh N, ψ() will also be compacly suppored on he inerval [,N ]. Daubechies consruced a family of H(z) wih impulse response lenghs N = 4,6,8,,... which saisfy he regulariy condiions. Moreover, her filers have he maximum possible number of zeros a ω = π, and hus are maximally regular (i.e., hey yield he smoohes possible φ() for a given suppor inerval). I urns ou ha hese filers are he maximally fla filers derived by Herrmann [3] long before filerbanks and waveles were in vogue. Below we show φ(), Φ(Ω), ψ(), and Ψ(Ω) for various members of he Daubechies wavele sysem. See [] for a more complee discussion of hese maers. db (N = 4): db3 (N = 6):.5 scaling fxn: φ(). scaling fxn CTFT: Φ(Ω).5 scaling fxn: φ(). scaling fxn CTFT: Φ(Ω) sec 4 6 Hz sec 4 6 Hz wavele: ψ() wavele CTFT: Ψ(Ω) wavele: ψ() wavele CTFT: Ψ(Ω) sec 4 6 Hz sec 4 6 Hz c P. Schnier, 3

32 db4 (N = 8): db5 (N = ):.5 scaling fxn: φ(). scaling fxn CTFT: Φ(Ω).5 scaling fxn: φ(). scaling fxn CTFT: Φ(Ω) sec 4 6 Hz sec 4 6 Hz.5 wavele: ψ() wavele CTFT: Ψ(Ω).5 wavele: ψ() wavele CTFT: Ψ(Ω) sec 4 6 Hz sec 4 6 Hz db7 (N = 4): db8 (N = 6):.5 scaling fxn: φ(). scaling fxn CTFT: Φ(Ω).5 scaling fxn: φ(). scaling fxn CTFT: Φ(Ω) sec 4 6 Hz sec 4 6 Hz.5 wavele: ψ() wavele CTFT: Ψ(Ω) wavele: ψ() wavele CTFT: Ψ(Ω) sec 4 6 Hz sec 4 6 Hz db9 (N = 8): db (N = ): scaling fxn: φ() scaling fxn CTFT: Φ(Ω) scaling fxn: φ(). scaling fxn CTFT: Φ(Ω) sec 4 6 Hz sec 4 6 Hz wavele: ψ() wavele CTFT: Ψ(Ω) wavele: ψ() wavele CTFT: Ψ(Ω) sec 4 6 Hz sec 4 6 Hz c P. Schnier, 3

33 Compuaion of φ() and ψ() The Cascade Algorihm: Given coefficiens {h[n]} ha saisfy he regulariy condiions, we can ieraively calculae samples of φ() on a fine grid of poins {} using he cascade algorihm. Once we have obained φ(), he wavele scaling equaion can be used o consruc ψ(). In his module we assume ha H(z) is causal wih impulse response lengh N. Recall, from our discussion of he regulariy condiions, ha his implies φ() will have compac suppor on he inerval [,N ]. The cascade algorihm is described below.. Consider he scaling funcion a ineger imes = m {...N }: φ(m) = N h[n]φ(m n). n= Knowing ha φ() = for / [,N ], he previous equaion can be wrien using an N N marix. In he case N = 4, we have φ() h[] φ() φ() φ() = h[] h[] h[] φ() h[3] h[] h[] φ(). φ(3) } {{ h[3] } φ(3) H The marix H is srucured as a row-decimaed convoluion marix. From he marix equaion above, we see ha [φ(),φ(),φ(),φ(3)] mus be (some scaled version of) he eigenvecor of H corresponding o eigenvalue ( ). In general, he nonzero values of {φ(n) : n Z}, i.e., [φ(),φ(),...,φ(n )], can be calculaed by appropriaely scaling he eigenvecor of he N N row-decimaed convoluion marix H corresponding o he eigenvalue ( ). I can be shown ha his eigenvecor mus be scaled so ha N n= φ(n) =.. Given {φ(n) : n Z}, we can use he scaling equaion o deermine {φ( n ) : n Z}: φ( m ) = N h[n]φ(m n). This produces he N non-zero samples {φ(),φ( ),φ(),φ(3 )...,φ(n )}.. Given {φ( n ) : n Z}, he scaling equaion can be used o find {φ(n 4 ) : n Z}: n= φ( m 4 ) = N h[n]φ( m n) n= = = p p even h[ p ]φ(m p ) h [p]φ[m p] where h [n] denoes he impulse response of H(z ), i.e., a -upsampled version of h[n], and where φ[m] = φ( m ). Noe ha {φ(n 4 ) : n Z} is he resul of convolving h [n] wih {φ[n]}. c P. Schnier, 33

34 3. Given {φ( n 4 ) : n Z}, anoher convoluion yields {φ(n 8 ) : n Z}: φ( m 8 ) = N h[n]φ( m 4 n) n= = p h 4 [p]φ[m p] 4 where h 4 [n] is a 4-upsampled version of h[n] and where φ[m] = φ( m 4 4 ). l. A he l h sage, {φ( n )} is calculaed by convolving he resul of he (l ) h sage l wih a l -upsampled version of h[n]: φ l [m] = p h l [p]φ l [m p] For l, his gives a very good approximaion of φ(). A his poin, you could verify he key properies of φ(), such as orhonormaliy and he saisfacion of he scaling equaion. Below we show seps hrough 4 of he cascade algorihm, as well as sep, using Daubechies db coefficiens (for which N = 4)..5 ieraion.5 ieraion ieraion.5 ieraion ieraion 4.5 ieraion c P. Schnier, 34

35 Finie-lengh Sequences and he DWT Marix: The wavele ransform, viewed from a filerbank perspecive, consiss of ieraed -channel analysis sages like he one below. c k [m] H(z ) G(z ) c k+ [n] d k+ [n] Firs consider a very long (i.e., pracically infinie-lengh) sequence {c k [m] : m Z}. For every pair of inpu samples {c k [n],c k [n ]} ha ener he k h filerbank sage, exacly one pair of oupu samples {c k+ [n],d k+ [n]} are generaed. In oher words, he number of oupus equal he number of inpus during a fixed ime inerval. This propery is convenien from a real-ime processing perspecive. For a shor sequence {c k [m], m =...M }, however, linear convoluion requires ha we make an assumpion abou he values of c k [m] for m / {...M }. One assumpion could be c k [m] = for m / {...M }. If we assume ha boh H(z ) and G(z ) have impulse response lenghs of N, and ha boh M and N are even, hen linear convoluion implies ha M nonzero inpus yield M+N oupus from each branch, for a oal of M+N > M oupus. The fac ha each filerbank sage produces more oupus han inpus could be inconvenien for many applicaions. A more convenien assumpion regarding he ails of {c k [m], m =...M } is ha he daa ouside of he ime window {...M } is a cyclic exension of daa inside he ime window: c k [m] = c k [ m M ] for m / {...M }. Recall ha a linear convoluion wih an M-cyclic signal is equivalen o a circular convoluion wih one M-sample segmen of he signal. Furhermore, he oupu of his circular convoluion is iself M-cyclic, implying our -downsampled branch oupus will be cyclic wih period M/. Thus, given an M-lengh inpu sequence, he filerbank oupu consiss of exacly M unique values. I is insrucive o wrie he circular-convoluion analysis filerbank operaion in marix form. Below we give an example for filer lengh N = 4, sequence lengh M = 8, and causal synhesis filers H(z) and G(z). c k+ [] c k+ [] c k+ [] c k+ [3] d k+ [] d k+ [] d k+ [] d k+ [3] }{{} ] [ ck+ = h[] h[] h[] h[3] c k [] h[] h[] h[] h[3] c k [] h[] h[] h[] h[3] c k [] h[] h[3] h[] h[] c k [3] g[] g[] g[] g[3] c k [4] g[] g[] g[] g[3] c k [5] g[] g[] g[] g[3] c k [6] g[] g[3] g[] g[] c k [7] }{{}}{{} ] c k [ HM d k+ G M c P. Schnier, 35