Contents Introduction 3 Multiresolution and scaling functions 4. The scaling equation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5..

Transcription

1 eort no. 95/3 Multiresolution and wavelets David Handscomb Wavelets may ossibly rovide a useful basis for the numerical solution of dierential or integral equations, in articular when they model rocesses which oerate on very dierent sace- or time-scales (turbulence, boundary layers, shocks, for instance). Our aim in this reort is to rovide a self-contained introduction to the basic theory of multiresolution and wavelets, accessible to the numerical analyst who knows nothing of signal rocessing, quantum theory, or any of the other established elds of alication. The author wishes to record his thanks in articular to Will Light and Gilbert Strang for showing him that wavelets are not as fearsome as they look. Oxford University Comuting Laboratory Numerical Analysis Grou Wolfson uilding Parks oad Oxford, England O 3QD dch@comlab.oxford.ac.uk October, 995

2 Contents Introduction 3 Multiresolution and scaling functions 4. The scaling equation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5.. The Fourier transform : : : : : : : : : : : : : : : : : : : : : : : : : 5.. The symbol of the scaling equation : : : : : : : : : : : : : : : : : : 7..3 Matrix reresentation of the scaling equation : : : : : : : : : : : : : 8. eroduction of olynomials : : : : : : : : : : : : : : : : : : : : : : : : : : 9.. eroduction of constants : : : : : : : : : : : : : : : : : : : : : : : 9.. eroduction of higher-degree olynomials : : : : : : : : : : : : : : 0.3 Continuity of the scaling function : : : : : : : : : : : : : : : : : : : : : : :.4 Autocorrelations of the scaling function : : : : : : : : : : : : : : : : : : : : 3.4. Orthogonality : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5 3 Wavelets 5 3. Sanning of comlementary sace by wavelets : : : : : : : : : : : : : : : : 7 3. Construction of wavelet coecients : : : : : : : : : : : : : : : : : : : : : : Dual scaling functions and wavelets : : : : : : : : : : : : : : : : : : : : : : The dual scaling function : : : : : : : : : : : : : : : : : : : : : : : The dual wavelet : : : : : : : : : : : : : : : : : : : : : : : : : : : : 3.4 Examles of scaling functions and wavelets and their duals : : : : : : : : : 3.4. Piecewise constant : : : : : : : : : : : : : : : : : : : : : : : : : : : 3.4. Piecewise linear : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Slines of higher order : : : : : : : : : : : : : : : : : : : : : : : : : 5 4 iorthogonal scaling functions and wavelets 5 4. Examles of biorthogonal scaling functions and wavelets : : : : : : : : : : 6 5 The orthogonal scaling functions and wavelets of Daubechies 9 6 Decomosition and reconstruction 3

3 - ) ) ) ) ) Introduction The essence of multiresolution analysis is the decomosition of a (continuous or discrete) signal or other function into comonents having diering fundamental scales in the indeendent variable (time or sace ). The same may be claimed for Fourier analysis; but there are these dierences: Fourier analysis can detect the resence of various frequencies in the signal as a whole, but cannot isolate the arts of the signal where these frequencies are signicant. Thus it is ne for analysing a steady signal, but not for a changing signal it can distinguish tone-colours but not tunes. Multiresolution analysis is less recise in icking out frequencies as we shall describe it, it can do little more than distinguish between octaves but is able to chart their variation with time. G I!!$ )! # " b r! " b! ) -!!)!$ -! )!)!$ -!!!$ )!!$ )!- b r!! -!!!$ ) -! )!)!$ -!!!$ ) [= 04 ; : : : ; 5;4 ] [= 03 ; : : : ; 73 ] [= 0 ; : : : ; 3 ] [= 0 ; ] [= = 00 ] Figure : An illustration to suggest the relationshis between the wavelets jk Wavelets are basis functions that may be used to erform such a multiresolution analysis. A wavelet is in a sense localised both in frequency and in time, normally taking a form such as jk(t) := ( k t j); (.) where is a so-called `mother' wavelet. Here the index j of jk secies its location in time and the index k its location in frequency a unit change in j reresents a shift of k units of time forwards or backwards while a unit change in k reresents a transosition of one octave u or down, or a dilation by a factor of or in the indeendent variable. This idea is roughly illustrated by Fig.. A kind of `Heisenberg uncertainty rincile' oerates [, Section 3.]. If b (!) denotes the Fourier transform of, if ( ) denotes the standard deviation of a robability distribution over time t with density roortional to j (t)j, and if (b ) denotes that of a rob- For most of this reort we shall refer to the indeendent variable as `time', but the wavelet rincile extends to functions of more than one variable in hotograhic image analysis, for instance where it is obviously more natural to refer to it as a vector in sace. Note: as well as the theory covered in this reort, which is founded on a countable wavelet basis of the form (.), there is another body of theory based on a double continuum of wavelets of the form (t) := ( t ) for any, and some suitable `mother' wavelet. Our feeling is that the latter context is less likely to have alications in numerical analysis than the one we shall treat here. 3

4 ability distribution over frequency! with density roortional to ( )(b ) : b (!), then Thus there is a fairly recise trade-o between recision in time location and recision in frequency resolution. We believe that wavelets were rst used in the analysis of seismograhic traces (see Morlet [6, 7]), although the fundamental rincile of time and frequency localisation is much older (see Gabor [0]). For further general background reading, we cite works by Chui [, ], Cohen and yan [4], Daubechies [7], Devore and Lucier [8], Meyer [4, 5] and Strang [8, 9]. Multiresolution and scaling functions We work throughout in the context of the sace L () of real-valued functions. Suose that we have an innite sequence of nested closed subsaces with the following roerties. S k V k is dense in L ();. T k V k = f0g; f0g V V 0 V V L () (.) 3. f( j) V 0 8j if and only if f() V 0 ; 4. f() V k+ if and only if f( ) V k 8k ; 5. 9 V 0 such that f( j) : j g is an unconditional basis 3 (or iesz basis) for V 0. Some writers insist on the functions f( j) : j g being orthogonal, or even orthonormal; we shall do so later, but not at this stage. Proerty ensures that any function in L () can be aroximated as closely as we lease by a function in V k, as long as we take k suciently large. Proerty is of minor imortance, for the resent. Proerty 3 is in fact a direct consequence of Proerty 5 and, together with Proerty 4, imlies that f( k j) V k if and only if f() V k. 3 This means that 9A > 0; < such that for every v V 0 9 unique coecients c `() (which must be real) such that and Also c `() imlies P c j( j) V 0. c j (t j) = v(t) 8t A kck kvk kck : 4

5 Proerty 4 imlies that the functions of any V k are dilations by a factor of k of the functions of V 0 their grahs are simly shrunk by that factor in the time direction. At the same time (.) imlies that every function of V 0 is also in V k for every k > 0. Proerty 5 leads us to dene as the scaling function associated with the nested saces (.). If we dene jk () := ( k j) (.) then, for each k, f jk : j g is an unconditional basis for V k. ecent work [9,,, 0] has generalised Proerty 5 so that it refers to a collection of more than one scaling function.. The scaling equation Since V 0 V and f( j) = j : j g is an unconditional basis for V, there must exist a unique set of real coecients `() (i.e. P j < ) such that (t) = j (t j) 8t : (.3) Equation (.3) is the scaling equation (alternative names: two-scale equation, renement equation, dilation equation,... ) satised by the scaling function... The Fourier transform The subsequent theory makes much use of Fourier transformation. Accordingly we here remind ourselves of some standard results (and incidentally settle on the normalisation we shall use). The validity of these results deends on certain conditions that we do not roose to discuss in detail here. Fourier series on the nite range [0; ] [The functions are assumed to have unit eriod.] Decomosition: econstruction: bf j := 0 f(t)e ijt dt; (.4) f(t) = bf j e ijt ; (.5) Parseval formula: Transformations: 0 jf(t)j dt = g(t) = f(t T ) () bg j = e ijt b fj ; g(t) = f 0 (t) () bg j = ij b fj ; 5 b fj ; (.6) (.7a) (.7b)

6 Convolution: h(t) = 0 f(s)g(s t)ds () b hj = b fj bg j : (.8) Continuity: [Here we shall quote only a coule of easily-roved results. Of course, a great deal more is known, and is still being discovered, in this area.] j fj b converges =) f(t) is Lischitz continuous 4 but, as only nearly a converse, f(t) is Lischitz continuous =) fj b = O(jjj ): (.9a) (.9b) Fourier transform on Decomosition: econstruction: Parseval formula: Transformations: Convolution: Continuity: h(t) = so that, in articular, bf(!) := f(t) = f(t)e i!t dt; (.0) jf(t)j dt = g(t) = f(t T ) () bg(!) = e i!t b f(!); bf(!)e i!t d!; (.) g(t) = f(t) () bg(!) = jj b f(!=); g(t) = f 0 (t) () bg(!) = i! b f(!); g(t) = tf(t) () bg(!) = ib f 0 (!); b f(!) d!; (.) (.3a) (.3b) (.3c) (.3d) f(s)g(s t)ds () b h(!) = b f(!)bg(!): (.4)! b f(!) d! exists =) f(t) is Lischitz continuous; (.5a) f(!) b = O(( + j!j) ); > 0 =) f(t) is Lischitz continuous: (.5b) 4 9 such that jf(s) f(t)j js tj. 6

7 The Poisson summation formula: We shall several times make use of the following: bf(j)e ijt : (.6) f(t ) = [This may be roved, assuming convergence, by noting that the left-hand side of (.6) is eriodic with eriod, and constructing the coecients of its Fourier series exansion.] A similar formula, in which the roles of f and b f are interchanged, which may be roved similarly, is: f(j)e ij! = bf(! ): (.7) Combining (.6) with the transformations of (.3) gives further results such as )f(t ) = ib f (t 0 (j)e ijt ; (.8) (t )f 0 (t ) =.. The symbol of the scaling equation Note that the transform of (.) is n bf(j) + j b f 0 (j)o e ijt : (.9) b jk (!) = k e ij!=k b (!= k ): (.0) or If we take the Fourier transform of the scaling equation (.3), we arrive at the equation b(!) = j e ij!= (!=) b (.) b(!) = P (e i!= )b (!=); (.) where we call the function P (z) := j z j (.3) the symbol of the scaling equation. When maniulating such symbols we shall assume always that jzj =, so that z = =z and (P having real coecients) P (z) = P (=z). Provided only that (0) b [= (t)dt] is nite and non-zero, (.) immediately gives us P () = ; j = : (.4) 7

8 For most of the time, we shall be considering the situation in which the coecients j vanish outside the range 0 j J (J being a ositive integer), when it is easy to show that has comact suort, vanishing outside the interval [0; J]. The scaling equation becomes and its symbol is the olynomial (t) = J j=0 P (z) =..3 Matrix reresentation of the scaling equation j (t j) (.5) J j=0 j z j : (.6) The values of the scaling function at the integers satisfy the innite system of linear equations (i) = j (i j) = i j (j): (.7) When has the comact suort [0; J], then this reduces to the nite system or (0) () () (3) (4). (J) C A = = P: J 0 C A@ (0) () () (3) (4). (J) C A (.8a) (.8b) Equations (.8) have a non-trivial solution if and only if P has as an eigenvalue; this solution then (unless 0 = or J = ) has (0) = (J) = 0, so that the scaling function vanishes at each end of its suort. If we can normalise this solution aroriately to give the values of at the integers, then recursive alication of the scaling equation enables us to generate values of at every intermediate oint of the form j k. [We could, if we liked, evaluate at other rational oints by the use of similar systems of equations, such as (=3) (=3) (4=3) (5=3) (7=3). (J =3) C A = J C (=3) (=3) (4=3) (5=3) (7=3). (J =3) C A ; (.9)

9 followed by further recursive alication of the scaling equation.] If the scaling function is everywhere r times dierentiable, then we may dierentiate the scaling equation (.3) u to r times, giving the further matrix equations (s) = s P (s) ; 0 s r: (.30) These equations have non-trivial solutions if and only if P has among its eigenvalues the values, =,..., = r. Of course, the matrix equations (.8), (.9) and (.30) determine the eigenvectors and (s) to within a scale factor only. Further analysis is required to determine the values and derivatives absolutely. We shall deal with this in Section... eroduction of olynomials Let P r denote the sace of all olynomials of degree r. Since P r 6 L (), we cannot actually have P r V k. It is ossible, however, for the basis of V k to san P r that is to say, it is ossible that for any f P r 9c [c 6 `() since P c j 6< ] such that f(t) = c j jk (t) 8t : (.3) Clearly if this is true for any k, say for k = 0, then it is true for all k. We shall then say that the sace V 0 or V k reroduces all olynomials in P r. eroduction of olynomials is worth aiming for, since it imlies that the error of the best aroximation in V k to a suciently smooth function is O( k(r+) )... eroduction of constants Suose that V 0 reroduces all constants [r = 0]. This will be the case if and only if 9c such that c j j0 (t) = c j (t j) = 8t : (.3) y alying the transformation t 7! t + to (.3), we see that c j must be indeendent of j. Since we have not yet normalised, we may choose to do so at this oint, and in such a way that c j, so that and, alying the Poisson summation formula (.6), (t j) = 8t (.33) b(j)e ijt = 8t (.34) so that b(j) = j0 : [Kronecker delta] (.35) 9

10 Now the Fourier-transformed scaling equation (.) gives us b(j) = P (e ij )b (j) = ( P ()b (j) (j even); P ( )b (j) (j odd): (.36) Equation (.35) will therefore hold if b (0) =, P () = (as already required by (.4)) and The last condition requires that or P ( ) = 0: (.37) ( ) j j = 0 (.38) j even j = j odd j = : (.39) If e denotes a vector of units, e T := ( ), then (.39) gives us e T P = e T, where P is the matrix of (.8) which therefore indeed has as an eigenvalue. We therefore can nd a corresonding right eigenvector to give the values of at the integers the normalisation (veried by substituting t = 0 in (.33)) being e T = : (.40) Note that we have shown this normalisation to be equivalent to the normalisation b(0) =.. eroduction of higher-degree olynomials (t)dt = : (.4) Now suose further that V 0 reroduces all linear functions [r = ], so that 9c such that c j (t j) = t 8t : (.4) y alying the transformation t 7! t + to (.4) and subtracting the original equation, we get (c j+ c j )(t j) = 8t ; (.43) so that (comaring (.33)) we must have c j = c 0 + j for some constant c 0. Thus j) c 0 g(t j) = ft c j g(t j) = 0 (.44) f(t 0

11 and, alying the Poisson formulae (.6) and (.8), or (since (.35) still holds) y dierentiating (.) we get fib 0 (j) c 0(j)ge b ijt 0 (.45) ib 0 (j) = c 0 b (j) = c0 j0 : (.46) b 0 (!) = i e i!= P 0 (e i!= )b (!=) + P (e i!= )b 0 (!=); (.47) so that using (.35) and (.37) we get (for j 6= 0) while b 0 (j) = ( b 0 (j) (j even); i P 0 (.48) ( )b (j) (j odd); b 0 (0) = ip 0 ()b (0): (.49) Equation (.46) will therefore hold rovided that in addition to b (0) =, P () = and P ( ) = 0 we have P 0 ( ) = 0; (.50) or and ( ) j j j = 0; (.5) c 0 = P 0 (): (.5) This extends to the general result that V 0 reroduces all olynomials of degree r if P () = and or P ( ) = P 0 ( ) = = P (r) ( ) = 0 (.53) ( ) j j s j = 0; s r; (.54) that is to say that P (z) is the roduct of a olynomial in z and the factor ( + z) r+. Let e s denote the vector whose transose is e T s := (0 s s s 3 s 4 s J s ), and let P P s := j even js j = j odd js j, 0 s r [ 0 = by (.39)]. Then (.54) gives us 0 0 C A P = = = =4 = C C e T 0 e T e T. e T r... r = r = r e T 0 e T e T. e T r A : (.55)

12 Therefore P has eigenvalues that include, =,..., = r, and (at least if these eigenvalues are simle) we can nd corresonding right eigenvectors,which will give derivatives u to the rth order of at the integers, rovided that we can normalise the eigenvectors correctly. We can do this by aealing to results like (.9) if we set f = and t = 0 this gives us the normalisation or The general normalisation is j 0 (j) = fb (j) + j b 0 (j)g = b (0) (.56) e T 0 = : (.57) e T s (s) = ( ) s s!; s r: (.58) [This normalisation is imossible when = s is a degenerate eigenvalue, with mutually orthogonal left and right eigenvectors.].3 Continuity of the scaling function If (.53) holds, then we can write r+ + z P (z) = P o (z); (.59) where P o is another olynomial with P o () =. From (.), b (!) = P (e i!= )b (!=), we then deduce that where b o (0) = and where e i! r+ b(!) = b o (!); (.60) i! b o (!) = P o (e i!= )b o (!=); (.6) since it is easily shown that, as the limit of an innite roduct, + e i!= + e i!=4 + e i!=8 = e i! : (.6) i! e i! r+ [In fact, as we shall see later, is the Fourier transform of the -sline i! of order r + with knots at the integers and suort (0; r + ).] Now e i! r+ = O(( + j!j) r ); (.63) i!

13 which is just not strong enough to enable us to use (.5) to rove the Lischitz continuity of the linear -sline (r = ). However, it enables us to say that (t) is Lischitz continuous if a sucient condition for which is that b o (!) = O(( + j!j) r ); > 0; (.64) jp o (z)j r 8z; jzj = : (.65).4 Autocorrelations of the scaling function If we write F (t) := then for real! we have, from (.4), (s)(s t)ds [= F ( t)] (.66) bf (!) = b (!) b (!) = b (!) (.67) and therefore (from the transformed scaling equation (.)) bf (!) = P (e i!= ) b F (!=): (.68) Note that (rovided that these exressions are meaningful) F (d) (t) = ( )d (s)(d) (s t)ds = ( ) r (d r) (s) (r) (s t)ds; 0 r d: (.69) From (.68) it follows that the function F is itself a scaling function 5, with the symbol S(z) := jp (z)j = P (z)p (=z) [= S(=z)]: (.70) The scaling equation for F thus has coecients s i := j i+j = s i : (.7) The function F is symmetrical, and if has suort [0; J] then F will have suort [ J; J] and the values and derivatives of F at the integers, when they exist, will form 5 In fact, any convolution of scaling functions is a scaling function; F () is the convolution of the two scaling functions () and ( ). 3

14 eigenvectors of the system or d F (d) ( J) ( J) F (d) F (d). ( ) F (d) (0) F (d) () F (d). (J ) (J) F (d) C A = s J s J s J s J s 0 s s s s s 0 s s 3 s s s s J s J s J s J 0 C F (d) ( J) ( J) F (d) F (d). ( ) F (d) (0) F (d) () F (d). C (J ) A (J) (.7a) F (d) C d f (d) = Sf (d) (.7b) with, clearly, F (J) = F ( J) = 0. If (.53) holds, P ( ) = P 0 ( ) = = P (r) ( ) = 0; (.73) then S( ) = S 0 ( ) = = S (r+) ( ) = 0; (.74) and we can show that S has eigenvalues, =,..., = r+. From (.33) and (.4) we nd that giving the normalisation F (j) = In general, the aroriate normalisation is (s) (s j)ds = (s)ds = ; (.75) e T f = : (.76) e T d f (d) = ( ) d d!; d r + ; (.77) where e T d := (( J)d ( ) d 0 d d d J d ). A similar technique may be used to evaluate integrals of roducts of more than two scaling functions, such as See [5, 3], for examle. (s)(s j)(s k)ds: 4

15 .4. Orthogonality As we mentioned earlier, we shall at a later stage want to introduce a condition that the basis functions f( j) : j g are mutually orthogonal, so that F (j) = 0 8j ; j 6= 0: (.78) Using the reverse form of the Poisson summation formula (.7), we see that (.78) holds if and only if the function P b F (! ) [a eriodic function with eriod ] is constant for all!. Now from (.68) we have ( P (e i!= ) bf F b (!= ) = S(e (! ) = i!= ) F b (!= ) ( even); P ( e i!= ) F b (!= ) = S( e i!= ) F b (!= ) ( odd): (.79) Hence, dening for jzj = (z) := bf (! ) = F (j)z j [= (=z)]; (.80) where! =!(z) is the solution in [0; ] of e i! = z, we have (z ) = P (z)p (=z) (z)+p ( z)p ( =z) ( z) = S(z) (z)+s( z) ( z): (.8) If is to be constant, therefore, we must have P (z)p (=z) + P ( z)p ( =z) = S(z) + S( z) = 8z : jzj = : (.8) In terms of the coecients, (.8) gives s 0 = and s j = 0; j 6= 0, or j = j i+j = 0; i 6= 0 (.83) as necessary conditions for orthogonality. Conversely, using the fact that b F (!), (z) and S(z) are all real and non-negative, we can show that (.8) imlies that ( ) = (). Suose, then, that (z) attains a maximum (or minimum) value M for some z with jzj =. Then (.8) imlies that both ( z) = M and ( z) = M. On the assumtion that is continuous at z =, reeating this rocess leads to a contradiction unless M = (). The necessary conditions (.8) or (.83) are therefore (almost) sucient conditions for orthogonality, as well. 3 Wavelets Now let W k denote the orthogonal comlement of the sace V k in the sace V k+, so that V k+ = V k W k. [See Fig..] Proerties and of V k on age 4 imly that 5

16 . L k W k is dense in L (), and the saces W k are easily shown to have the roerties. f( j) W 0 if and only if f() W 0 8j ; 3. f() W k+ if and only if f( ) W k 8k ; corresonding to Proerties 3 and 4 of V k, so that f( k j) W k if and only if f() W k, and the functions of any W k are dilations by a factor of k of the functions of W 0. However, W k is a subsace not of W k but of V k all of the saces fw k g are mutually orthogonal.! V 0! V! V!! V k! V k % % % % % % W 0 W W W k W k Figure : Inclusion and comlementarity relationshis between the saces fv k g and fw k g Any v K V K now has a sequence of orthogonal decomositions v K = w K + v K = w K + w K + v K = = w K + + w + v = w K + + w + w 0 + v 0 = ; (3.) where v k V k and w k W k, and a general f L () has the sequence of orthogonal decomositions f = v k + w j = w k : (3.) j=k k We now dene a wavelet 6 (that is, a `mother wavelet', in the terminology of Section ; the scaling function has been called the `father wavelet') to be a function W 0 (if one exists) such that f ( j) : j g is an unconditional basis for W 0 (and so generates W 0 and W k in the same way as generates V 0 and V k ). Since V 0? W 0, we must have and since V 0 V k? W k and W 0 V k? W k (k > 0) we have (t) (k t j)dt = (t) (t j)dt = 0; j ; (3.3) (t) (k t j)dt = 0; k > 0; k ; j : (3.4) 6 y some writers, articularly of the French school, such a function would be called only a `re-wavelet' (`re-ondelette') to become a wavelet (`ondelette') it would need further to be made orthonormal to its integer translates: (t) (t j)dt = j0; j. See Section 5. 6

17 Since W 0 V, there exists another unique set of coecients q `() such that (t) = q j (t j) 8t : (3.5) This is a two-scale relationshi between and, similar to the scaling equation (.3), which may similarly be exressed using Fourier transforms: where ^(!) = Q(e i!= ) ^(!=) (3.6) Q(z) := q j z j : (3.7) The function, the coecients q and the function Q are not uniquely determined by and P, but we can deduce from (3.3) various relationshis that must hold between them, and can thus construct one ossibility. Dene F (t) := (s)(s t)ds (3.8) so that we require F (j) = 0 8j, in order to satisfy (3.3). Then for real! we have, using (.4), bf (!) = b (!) b (!) (3.9) so that bf (!) = Q(e i!= )P (e i!= ) b F (!=): (3.0) Using (.7) again, we nd that we need (z) := P b F (! ) to vanish identically for jzj =, whereuon (3.0) gives us the condition (z ) Q(z)P (=z) (z) + Q( z)p ( =z) ( z) = 0; 8z : jzj = : (3.) 3. Sanning of comlementary sace by wavelets If we can dene one accetable W 0, then there cannot be a third function, V say, orthogonal to all integer translates of both and. For suose that there exists such a. Since V, there exist coecients r `() such that (t) = r j (t j); giving b(!) = (e i!= )b (!=): Hence we deduce bf (!) = (e i!= )P (e i!= ); (3.) bf (!) = Q(e i!= )(e i!= ); (3.3) 7

18 leading to the orthogonality conditions (z ) (z)p (=z) (z) + ( z)p ( =z) ( z) = 0; 8z : jzj = ; (3.4) (z ) Q(z)(=z) (z) + Q( z)( =z) ( z) = 0; 8z : jzj = : (3.5) Combining (3.) and (3.5), we deduce (for almost all z, jzj = ) (z)=p (z) = ( z)=p ( z); whence, substituting into (3.4) and using (.8), we get (z) P (z) fp (z)p (=z) (z) + P ( z)p ( =z) ( z)g = (z) P (z) (z ) = 0: This must imly that (z) 0, so that (t) 0, and there is no function in V orthogonal to all integer translates of and. We deduce that the translates of and san V, or equivalently that the translates of san W Construction of wavelet coecients Equation (3.39) suggests that we take Q(z) = zp ( =z) ( z) (3.6a) or q j = ( ) j k k F (j k) (3.6b) to obtain suitable coecients for the scaling equation (3.5). In adoting (3.6), we have in eect normalised, although we may not be able to describe this normalisation exlicitly in simle terms. If has the comact suort [0; J] then (z) = J j= J F (j)z j = (z J z z 0 z z z J )f (3.7) where f is the normalised eigenvector corresonding to the unit eigenvalue of the matrix S in (.7) (with F ( J) = F (J) = 0). The scaling equation (3.5) will have 3J non-zero coecients q j, j running from ( J) to J, and the wavelet will consequently have the comact suort [ J; J], of length J. 3.3 Dual scaling functions and wavelets If we further decomose every term of the summation (3.) in terms of the aroriate basis w k = d jk jk (3.8) 8

19 where jk (t) = ( k t j) as in (.), then we have a comlete least-squares decomosition of f in terms of wavelets f = d jk jk : (3.9) k Such a decomosition can be used as a basis for signal comression. Since every basis function jk has the same maximum modulus, we can set a threshold level and discard every term for which jd jk j falls below this threshold, with the condence that this will not significantly alter the shae of f. The roblem is how to determine d jk. There is no diculty in case the wavelets ( j) are mutually orthogonal since we then simly have d jk = f(t) jk(t)dt. jk (t) dt; (3.0) which is relatively easy to comute. This holds in the case that we shall discuss in Section 5. We may, however, want more freedom in our choice of wavelets. In this case, one answer is to construct dual wavelets e jk W k such that jk (t)e j 0 k 0(t)dt = 0 unless j = j0 and k = k 0 : (3.) Then d jk = f(t) e jk (t)dt. jk (t)e jk (t)dt: (3.) These dual wavelets will be dilated translates of a dual mother wavelet e W The dual scaling function efore constructing the dual wavelet e, we construct a dual scaling function e V0 such that F (j) := ~ e(t)(t j)dt = j0 : (3.3) Since e V0, there exist unique coecients c `() such that e (t) = P c j(t j), and we can easily show that c j = e (t)e (t j)dt = F ~ (j), so that e(t) = F ~ (j)(t j): (3.4) Similarly, we can show that (t) = F (j)e (t j): (3.5) Alying (3.3) to (3.4), we get F (j)f ~ (k j) = k0 (3.6) 9

20 so that we must have ~ (z) = = (z): (3.7) Since (because of (3.5)) the translates of e form a basis for V0 and its dilated translates form a basis for V, there is a dual scaling equation e(t) = e j e (t j) (3.8) with symbol e P (z). Following an argument similar to one used earlier, we have bf ~ (!) = b e (!) b (!) = P e (e i!= )P (e i!= ) F b ~ (!=) (3.9) ~ (z ) = e P (z)p (=z) ~ (z) + e P ( z)p ( =z) ~ ( z) = 8z; jzj = ; (3.30) so that ep (z)p (=z) + e P ( z)p ( =z) = 8z; jzj = : (3.3) Comaring (.8), we see that this will be satised if ep (z) = P (z) (z) : (3.3) (z ) Hence we may obtain the coecients of the dual scaling equation, as follows. We notice that if has comact suort, so that P (z) is a olynomial of nite degree, then e P (z) will be a rational function of z [unless is constant as in the case of orthogonal scaling functions], so that the dual scaling function e will have innite suort. In order to obtain the coecients of the dual scaling equation, we exand (3.3) in a convergent ower series. We can do so by factorising, rovided that the roots of are distinct and none lies on the unit circle, for suose that (z) [= (=z)] = c Y i fz + ( i + = i ) + =zg (3.33) where j i j <, c = Q i i=( + i ). Then we can use the artial fraction decomosition to write ( )( =) fz + ( + =) + =zgfz + ( + =) + =zg = = z + ( + =) + =z z + ( + =) + =z (3.34) (z) = ~ (z) = Q c fz + ( i i + = i ) + =zg Qj6=i = f( j i )( = j i )g : (3.35) cfz + ( i + = i ) + =zg i 0

21 Each individual term of this sum is then converted into a Laurent exansion (valid as required on jzj =, since we have chosen to take j i j < ) fz + ( i + = i ) + =zg = i f i (z + =z) + i (z + =z ) g: (3.36) i Substituting from (3.36) into (3.35) and thence into the denominator of (3.3), we obtain ep (z) as a Laurent ower series. More usefully, erhas, the coecients of the ower series obtained by substituting from (3.36) into (3.35) are the coecients F ~ (j) in the formula (3.4) exressing e in terms of The dual wavelet Having constructed e P (z), we derive a ossible corresonding e Q(z) by alying (3.6a) thus: eq(z) = z e P ( =z) ~ and again use (3.35) to convert this into a Laurent ower series. We may note the relationshis or ep (z)p (=z) + e P ( z)p ( =z) = ; ep (z)q(=z) + e P ( z)q( =z) = 0; eq(z)p (=z) + e Q( z)p ( =z) = 0; eq(z)q(=z) + e Q( z)q( =z) = j j e j i+j = j e j q i+j = j ( =z) ( z) = zp ; (3.37) (z ) eq j q i+j = i0 ; eq j i+j = 0: (3.38a) (3.38b) (3.38c) (3.38d) (3.39a) (3.39b) The dual mother wavelet may be derived from the dual scaling function by use of the scaling equation e (t) = eq j(t e j) (3.40) (where Q(z) e P = eq j z j ) or else (erhas more conveniently in some cases, since (3.40) involves an innite summation of an innite summation) directly from the original scaling function or wavelet e (t) = q (t j) = j r j (t j) (3.4) with Q zp ( =z) (z) = (z) (z ) ; (z) = (z) ( z) (z ) : (3.4) From (3.38d) it can be shown that the wavelets and their duals satisfy (3.) as required.

22 3.4 Examles of scaling functions and wavelets and their duals 3.4. Piecewise constant The simlest examle 7 of a scaling function [with J = ] is the Haar basis function (t) := (0 t < ); 0 (elsewhere); (3.43) which satises the scaling equation (t) = (t) + (t ) (3.44) so that 0 = = and Its Fourier transform is P (z) = + z : (3.45) b(!) = e i! : (3.46) i! Figure 3: Haar scaling function and wavelet V 0 is then the sace of iecewise-constant functions with discontinuities at the integers, which reroduces all constants. P is the unit matrix P = ; (3.47) whose reeated eigenvalue of reects the fact that (0) and () are indeterminate. The matrix of (.9) is 0 ; (3.48) 0 with eigenvalues. F (t) is the `hat' function F (t) = 8 < : + t ( t < 0); t (0 t < ); 0 (elsewhere); (3.49) 7 aart from the trivial case of the delta function which may be thought of as a scaling function with J = 0, satisfying the equation (t) = (t), but which does not lead to a wavelet

23 We have s 0 =, s = s = and S(z) = ( + z) =4z. S is the 3 3 matrix S = and (z). For the corresonding wavelet, we have = = = = A (3.50) so that q 0 =, q = and Q(z) = z (t) = (t) (t ) 8 < (0 t < ); = ( : t < ); 0 (elsewhere): (3.5) (3.5) This scaling function and wavelet are (trivially) orthogonal to all of their integer translates, and are self-dual. They are shown in Fig Piecewise linear A more interesting examle of a scaling function is the hat function [J = ] (t) := 8 < : t (0 t < ); t ( t < ); 0 (elsewhere); (3.53) satisfying the scaling equation so that 0 = =, = and Its Fourier transform is (t) = (t) + (t ) + (t ) (3.54) + z P (z) = : (3.55) e i! b(!) = : (3.56) i! V 0 is the sace of continuous iecewise-linear functions with gradient discontinuities at the integers, reroducing all linear olynomials. P is the 3 3 matrix P = = = = = 3 A ; (3.57)

24 @ Figure 4: `Hat' scaling function and wavelet e e Figure 5: Duals of scaling function and wavelet in Fig. 4 (truncated in! direction) with a simle eigenvalue of and a reeated eigenvalue of reecting the fact that (0), () and () are well-dened but 0 (0), 0 () and 0 () are indeterminate and F (t) is now a cubic -sline on the interval [ ; ]. The matrix of (.9) is 0 = 0 = = 0 = 0 C A ; (3.58) with eigenvalues,. We have s 0 = 3 4, s = s =, s = s = 8 and S(z) = (+z)4 =6z. The matrix S is S = =8 3=4 = =8 =8 = 3=4 = =8 =8 = 3=4 =8 and (z) = (z =z)=6. The symbol of the corresonding wavelet relationshi is giving q = q =, q = q =, q 0 = 5 6. Q(z) = z 6z + 0 6=z + =z ; (3.59) 4 4 C A

25 Using (3.36) we have ~ (z) = (z) = 3f ( 3)(z + =z) + ( 3) (z + =z ) g; (3.60) from which we may derive the dual scaling function and wavelet. The functions and, which have comact suort, are shown in Fig. 4, while e and e are shown in Fig. 5. Although both of the dual functions are suorted on the whole real line, it can be seen that they fall o very raidly (in fact exonentially) to zero in both directions Slines of higher order These examles are easily generalised: the scaling equation with symbol J + z P (z) = (3.6) generates a scaling function with Fourier transform e i! J b(!) = (3.6) i! which is simly the normalised -sline of order J with knots at the integers 0,,..., J (the J-fold convolution of (3.46). The sace V 0 is then the sace of sline functions of order J, reroducing all olynomials u to degree J. The corresonding sline wavelet and the duals e and e are not hard to nd. In fact it can be shown that, for each order J > 0, the sace W 0 is the sace of sline functions of the form w(t) = dj s(t) dt J ; where s(t) is a sline function of order J (iecewise olynomial of degree J ) with knots at the integers and half-integers, vanishing at the integers [s(j) = 0 8j ]. 4 iorthogonal scaling functions and wavelets It is somewhat inconvenient that if and have comact suort then their duals e and e, as dened in Section 3.3, must have innite suort. It is often ossible to avoid this diculty by going over to biorthogonal multiresolution [3]. Starting with the same nested sequence (.) of subsaces V V 0 V V L () (4.a) and their associated scaling function, we attemt to dene three further sequences of subsaces ; W V 0 ; W 0 V ; W V ; ; (4.b) 5

26 e V e V0 e V e V L (); ; f W e V0 ; f W0 e V ; f W e V ; ; (4.c) (4.d) each having the translation and scaling roerties 3 and 4 of Section, and with V k+ = V k W k and e Vk+ = e Vk f Wk. However, these two decomositions are now to be not orthogonal (V k 6? W k and e Vk 6? f Wk ) but biorthogonal, in the sense that V k? f Wk ; e Vk? W k : (4.) More recisely, we ask that the saces W 0, e V0 and f W0 be sanned by integer translates of functions, e and e with the roerties that (s) e (s j)ds = (s) e (s j)ds = j0 8j ; (4.3a) (s) e (s j)ds = (s) e (s j)ds = 0 8j : (4.3b) As well as the scaling equations (.3) and (3.5) for and, with symbols P (z) and Q(z), there will again be the further scaling equations (3.8) and (3.40) with symbols e P (z) and e Q(z), and the biorthogonality conditions (4.3) will lead to relationshis ep (z)p (=z) + P e ( z)p ( =z) = ; ep (z)q(=z) + P e ( z)q( =z) = 0; eq(z)p (=z) + Q( z)p e ( =z) = 0; eq(z)q(=z) + Q( z)q( =z) e = (4.4a) (4.4b) (4.4c) (4.4d) exactly like those (3.38) that hold between the symbols of dual scaling equations. This time, however, e P and e Q are no longer forced to take the forms (3.3) and (3.37), and consequently we may be able to arrange for all four symbols to have nitely many non-zero coecients i.e. to be olynomials in z ossibly divided by owers of z. Given any P such that P () =, P ( ) = 0, there may in fact be many ossible e P such that e P () = and (4.4a) holds. Taking any such e P, if we then let Q(z) = z P e ( =z); eq(z) = zp ( =z); (4.5a) (4.5b) then it is easy to see that all four of the equations (4.4) will hold. 4. Examles of biorthogonal scaling functions and wavelets For examle, take the iecewise-linear (`hat') scaling function of Section 3.4. with symbol (3.55) + z + z + z P (z) = = : (4.6) 4 6

27 Then one way to satisfy (4.4) is to take (4.5) then gives us However, ep (z) = =z + + 6z + z z z =z + 4 z = ; (4.7) Q(z) = =z =z + 6 z z 8 (4.8a) eq(z) = =z + z : 4 (4.8b) Figure 6: iorthogonal scaling functions (left) and wavelets (right) generated by (4.7) and (4.8) su =z + 4 z = 3; jzj= from which we can show that condition (.65) is far from satised. In fact, the scaling function e aears to be continuous (see Fig.6). However, in generating this gure, we were unable to use (.8a) since the unit eigenvalue of the matrix P = =4 3= = =4 =4 = 3= = =4 =4 = 3= =4 is degenerate. Instead, we made use of (.9) to obtain the values of e at the oints, 3,,, &c, and then used the scaling equation to evaluate it at intermediate oints of the form j k =3, where j is not a multile of 3. These are the values that are lotted in Fig.6. 7 C A

28 We can obtain a smoother e by increasing the extent of its suort. If we take ep (z) = =z3 =z 4=z z + 0z 4z 3 z 4 + z z =z 3 4=z + 3=z z 4z + z 3 = ; (4.9) 8 and we have Q(z) = =z4 + =z 3 4=z 0=z + 0z 4z + z 3 + z 4 3 (4.0a) eq(z) = =z + z ; 4 (4.0b) su =z3 4=z + 3=z z 4z + z 3 8 = : jzj= This still does not allow us to satisfy (.65); this e P does, however, yield a continuous scal- Figure 7: iorthogonal scaling functions (left) and wavelets (right) generated by (4.9) and (4.0) ing function e (see Fig.7). Note that (4.7) and (4.8) give wavelets and e with suorts of length 3, while (4.9) and (4.0) give wavelets with suorts of length 5. 8

29 5 The orthogonal scaling functions and wavelets of Daubechies As we remarked at the beginning of Section 3.3, the task of decomosing a function into wavelets is rendered much easier if the wavelets are both orthogonal and of comact suort. In Section.4. we saw that the scaling function is orthogonal to all of its integer translates if (z) ; P (z)p (=z) + P ( z)p ( =z) = : (5.) Dening and F (t) := (s) (s t)ds (5.) (z) = F (j)z j ; (5.3) we nd from the scaling equation (3.5) that bf (!) = Q(e i!= )Q(e i!= ) b F (!=) (5.4) so that (z ) = Q(z)Q(=z) (z) + Q( z)q( =z) ( z): (5.5) If (5.) holds, therefore, and we follow (3.6) in dening and so Q(z) = zp ( =z) ( z) = zp ( =z) q j = ( ) j j ; (5.6a) (5.6b) then we get (z ) = P ( =z)p ( z) + P (=z)p (z) = (5.7) so that the wavelet too is orthogonal to all of its integer translates. We therefore look for coecients 0,,... J, for some nite J, such that (.4) J j = ; (5.8a) j=0 such that (.54) J ( ) s j s j = 0; 0 s r (5.8b) j=0 for r as large as ossible, and such that (.83) J i j=0 j i+j = 0; 0 < i J: (5.8c) 9

30 If J is even, however, (5.8c) with i = J immediately gives us 0 J = 0, so that J eectively becomes J. Without loss of generality therefore, we may assume J to be odd. The scaling function cannot be either symmetrical or antisymmetrical (unless J =, when it is the Haar function of Section 3.4.) since, if it is, we can show that we must have J j = j 8j, or P (z) = z J P (=z); (5.9) when (5.) tells us that P is a olynomial of odd degree J such that P (z) P ( z) = (P (z) + P ( z))(p (z) P ( z)) = z J : (5.0) The only solution with P () = is P (z) = ( + z J )=, which yields an orthogonal scaling function only if J =. Equations (5.8) do, however, have useful solutions. The best-known are those resented by Daubechies in [6] in articular the solution with J = 3 and r = where + z so that P (z) = 0 = = z + 3( z) ; = ; 4 ; (5.) ; 3 = 3 : (5.) 4 Since this yields an orthogonal scaling function, we get a corresonding orthogonal wavelet by taking q = 0 ; q 0 = ; q = ; q = 3 : (5.3) Figure 8: The simlest Daubechies scaling function and wavelet The resulting functions are shown in Fig. 8. dierentiable 8, desite which generates a sace V 0 They are continuous but nowhere that reroduces all linear functions, 8 For a discussion of the analytic roerties of these functions, see the article by Pollen in [,. 3{3]. 30

31 since the matrix P = C A = C A (5.4) has eigenvalues that include and. y increasing the value of J, Daubechies and others have constructed further orthogonal scaling functions and wavelets, reroducing olynomials of higher degrees. The coef- cients in their scaling equations are, however, not generally exressible in a simle form. 6 Decomosition and reconstruction Suose that v k+ V k+, so that v k+ = c j;k+ j;k+ : (6.) Since V k+ = V k W k, v k+ has an orthogonal decomosition v k+ = v k + w k (6.) where v k = c jk jk V k ; (6.3a) w k = d jk jk W k : (6.3b) P P The scaling equations (.3) and (3.5) give us jk (t) = l l j+l;k+ (t) and jk (t) = l q l j+l;k+ (t). We deduce immediately that the coecients in (6.) can be derived from those of (6.3) by the relationshi c j;k+ = lfc lk j l + d lk q j l g: (6.4) Equation (6.4) is the basis of the reconstruction algorithm. To go in the reverse direction, we may make use of the dual functions jk e and e jk. Since (t) (t)dt e = (t) e (t)dt =, we have Therefore c lk = jk(t)e jk (t)dt = v k+(t)e lk (t)dt lk(t)e lk (t)dt jk (t)e jk (t)dt = k : (6.5) 3

32 d lk = = k v k+(t)e lk (t)dt = k c j;k+ j;k+ (t) e j lj;k+ e (t)dt = c j;k+ e j l ; v k+(t)e lk (t)dt lk(t)e lk (t)dt = k v k+(t)e lk (t)dt = k c j;k+ j;k+ (t) eq j lj;k+ e (t)dt = c j;k+ eq j l : Equations (6.6) are the basis of the decomosition algorithm. In summary, we have decomosition and reconstruction algorithms dened by c lk = P P c j;k+e j l d lk = P c j;k+eq j l c j;k+ = l fc lk j l + d lk q j l g (6.6a) (6.6b) (6.7) The decomosition and reconstruction algorithms can be shown to be `mutually inverse', by virtue of equations (3.39) that is to say, any number of decomosition stes followed by the same number of reconstruction stes, or any number of reconstruction stes followed by the same number of decomosition stes, gets us back to where we started. The foregoing statement, however, imlicitly assumes that we have a comlete innite sequence of data to work on; i.e. that the j sux in c jk and d jk runs from to +. This is rarely the case in ractical alications even if we can say what haened throughout the ast, we cannot know what is to haen in even the near future. Thus in ractice we can erform the reconstruction (6.4) exactly only if and have comact suort) and j is not too close to either end of the range of available data; similarly we can erform the decomosition (6.6) exactly only if e and e have comact suort and l is not too close to either end of the range. In ractice, consequently, decomosition and reconstruction are mutual inverses only in cases like those of Section 4 where,, e and e all have comact suort, or of Section 5 where and have comact suort and are self-dual. Even in these cases the mutual inverse roerty breaks down near either end of the range. [It is ossible to treat a nite sequence of data as eectively innite by treating it as eriodic, with the rst observation following on from the last rovided that the number of observations is a multile of K for K levels of decomosition.] 3

33 eferences [] C. K. Chui, 99. An Introduction to Wavelets. Academic Press. [] C. K. Chui, editor, 99. Wavelets: a Tutorial in Theory and Alications. Academic Press. [3] A. Cohen, I. Daubechies and J. C. Feauveau, 99. iorthogonal bases of comactly suorted wavelets. Comm. Pur. & Al. Math. [4] A. Cohen and. D. yan, 995. Wavelets and Multiscale Signal Processing. Chaman & Hall. (English version of 99 French ublication). [5] W. Dahmen and C. A. Micchelli, 993. Using the renement equation for evaluating integrals of wavelets. SIAM J. Num. Anal., 30: 507{537. [6] I. Daubechies, 988. Orthonormal bases of comactly suorted wavelets. Comm. Pur. & Al. Math., 4: 909{996. [7] I. Daubechies, 99. Ten Lectures on Wavelets. CMS/NSF Series in Al. Math. SIAM. [8]. A. DeVore and. J. Lucier, 99. Wavelets. Acta Numerica 99, ages {56. [9] G. Donovan, J. S. Geronimo, D. P. Hardin and P.. Massoust. Construction of orthogonal wavelets using fractal interolation functions. (to aear). [0] D. Gabor, 946. Theory of communication. J. IEE, 93: 49{457. [] J. S. Geronimo, D. P. Hardin and P.. Massoust, 994. Fractal functions and wavelet exansions based on several scaling functions. J. Arox. Theory, 78: 373{40. [] T. N. T. Goodman and S. L. Lee, 994. Wavelets of multilicity r. Trans. Amer. Math. Soc., 34: 307{34. [3] A. Kunoth, 995. Comuting renable integrals documentation of the rogram Version.. (to aear; available as PostScrit le on Web from htt:// [4] Y. Meyer, 990. Ondelettes et Oerateurs. Hermann. (3 volumes English translation of Vol. ublished as Wavelets and Oerators, Cambridge, 99). [5] Y. Meyer, 993. eview of [] and [7]. ull. Amer. Math. Soc., 8: 350{360. [6] J. Morlet, 98. Paer given at a geohysics conference. [7] J. Morlet, G. Arens, I. Fourgeau and D. Girard, 98. Geohysics, 47: 03{36. [8] G. Strang, 989. Wavelets and dilation equations: a brief introduction. SIAM ev., 3: 64{67. 33

34 [9] G. Strang, 993. Wavelet transforms versus Fourier transforms. ull. Amer. Math. Soc., 8: 88{305. [0] G. Strang and V. Strela, 995. Short wavelets and matrix dilation equations. IEEE Trans. in Signal Processing, 43: 08{5. 34