Wavelets and Approximation Theory. Bradley J. Lucier October 23, 2013 Revision: 1.37

Transcription

1 Wavelets and Approximation Theory Bradley J. Lucier October 23, 203 Revision:.37 These notes are incomplet and inkorrect(as an old computer documentation joke goes). Nonetheless, I ve decided to distribute them in case they prove useful to someone. My goal is to present certain results that can be proved in a (relatively) straightforward way. So, for some problems we present complete proofs in L 2 and partial proofs (or none at all) for L p, p 2. To simplify arguments even more, we consider only wavelet spaces of piecewise constant functions on uniform grids of size 2 k 2 k on the unit suare. As far as I know, all results can be extended to approximations of higher order. We don t consider at all the area of construction of wavelets, which is a large topic in itself. I ve presented some of these results in graduate topics courses at Purdue University. The notes have certain themes: generalized wavelets (results can be generalized to box splines, for example), nonlinearity (nonlinear approximations, nonlinear wavelet decompositions, avoiding arguments that use linear functionals, etc.), non-hilbert spaces (L p and l p for p 2), nonconvexity (L p and l p for 0 < p < ). My approach has been inspired by and is derived from the approach of Ron DeVore, who has made tremendous contributions to this field and with whom I collaborated for about a decade. Some parts of these notes are influenced directly by the survey paper Wavelets, by Ronald A. DeVore and Bradley J. Lucier, Acta Numerica (992), 56. Specific examples of nonlinear, piecewise constant, wavelet approximations appeared in Image compression through wavelet transform coding, by Ronald A. DeVore, Bjorn Jawerth, and Bradley J. Lucier, IEEE Transactions on Information Theory, 38 (992), And the approach in that paper was in turn influenced by the proofs in Interpolation of Besov spaces, by Ronald A. DeVore and Vasil A. Popov, Transactions of the American Mathematical Society 305 (988), But my goal is not to document everything that influenced these notes, for two reasons: () I can t remember where many ideas came from and (2) nearly all these results are now over 20 years old, so can appear, perhaps, in these informal notes without strict attribution. Please me at lucier@math.purdue.edu to get the latest version of these notes, or to point out errors or gaps in arguments. I thank Jeffrey Gaither, who helped correct and complete some aspects of these notes.

2 Background We use the following terminology. A norm on a vector space X, : X [0, ) R, satisfies () ( a R) ( x X) ax = a x, (2) ( x,y X) x+y x + y, and (3) ( x X x = 0) x = 0. A semi-norm satisfies () and (2), but not necessarily (3). A uasi-norm satisfies (), (3), and (2 ): ( C > ) ( x,y X) x+y C( x + y ). And a semi-uasi-norm (or uasi-semi-norm) satisfies () and (2 ). Elements of the seuence space { l p = x = (x,x 2,...) x k R, x p l p := } x k p < for 0 < p < (with the usual change when p = ) are denoted by (x k ) or {x k }. The notation A(f) B(f), A,B: X [0, ) R, means that there exist positive constants c, c such that for all f X, ca(f) B(f) ca(f). WesaythatAiseuivalent tob onx. Itisastandardresultthatallpairsof(uasi-)norms on a finite-dimensional space are euivalent. We use the following two ineualities extensively. For any bounded domain Ω R 2, let Ω be the volume of Ω. If 0 < < p <, we can use Hölder s ineuality on Ω with to see that for any we have Ω Ω f L p (Ω) := s = p and r + s = { f : Ω R f p L p (Ω) := Ω } f p < ( f ) ( ( ) /s ( ) /r ( f s) ( r = ) ) /p f p/. Ω Ω Ω Ω Ω Ω Taking th roots shows that ( () Ω Ω f ) / ( ) /p f p. Ω Ω On the other hand, for 0 < p and seuence norms we have (2) (x k ) lp (x k ) l ; when p = this is obvious, while for other values of p this ineuality will be proved later. 2

3 Nested finite-dimensional linear spaces Let I be the unit interval [0,] 2 ; for k 0 and j Z k 2 := {j = (j,j 2 ) Z 2 0 j,j 2 < 2 k } let be the suare with opposite corners j/2 k and (j +(,))/2 k ; in other words = 2 k (I +j). The corners of the suares and the suares themselves are called dyadic, as the corners are pairs of rational numbers whose denominators are powers of 2. We define the characteristic function of to be {, x Ij,k, χ j,k (x) := 0, otherwise. We want to project a function f defined on I onto functions that are constant on each. We ll call the space of such functions S k = { f : I R f Ij,k is a constant }. These spaces have the property that S k S k+, i.e., this is an expanding seuences of finite-dimensional spaces. Projections as near-best approximations For now, let s take P k f Ij,k = f, i.e., P k f in is the average of f on. P k f is the best L 2 (I) approximation to f from S k, as calculus shows. We make a series of claims about P k. First, note that P k f is bounded on L p (I) for p, because, for each, by (), P k f = f ( ) /p f p. So, P k f p 3 f p = f p ;

4 summing this ineuality over j and taking pth roots gives P k f Lp (I) f Lp (I). Second, if S S k, then P k S = S (which is obvious). Third, P k f is additive with respect to elements of S k : for any S in S k, P k (f +S) = P k f +P k S = P k f +S, S S k. P k is in fact linear, but we only need additivity. These three properties imply that P k f is a near-best approximationto f in S k, because for any S S k, we have P k f f Lp (I) = (P k f P k S)+(S f) Lp (I) P k f P k S Lp (I) + f S Lp (I) So for all f L p (I), f S Lp (I) + f S Lp (I). (3) P k f f Lp (I) 2 inf S S k f S Lp (I). Thesamewouldbetrueofany(possiblynonlinear)projector ofl p (I)ontoS k thatsatisfies these three properties. Moduli of smoothness We want to compare how close f is to P k f by considering the smoothness of f. So for any x I, nonnegative integer r, and h R 2 for which this makes sense, we define the differences Thus by induction, 0 hf(x) := f(x), r+ h f(x) := r hf(x+h) r hf(x), r 0. h f(x) = f(x+h) f(x), 2 hf(x) = f(x+2h) 2f(x+h)+f(x), etc.; r h f(x) = r s=0 We have x+sh I for s = 0,...,r, only for x in ( ) r ( ) r+s f(x+sh). s I rh := {x I x+rh I}, so for f(x) defined for x I, r h f(x) is defined for x I rh. 4

5 For any 0 < p, the rth modulus of smoothness in L p (I) is defined by ω r (f,t) p := sup r hf Lp (I rh ). h <t Bounding approximation error by modulus of smoothness We claim that for p, we have f P k f Lp (I) (2π) /p ω (f,2 k 2) p. Why? Let s consider p <. On each, we have f(x) P k f(x) p dx = f(x) f(y)dy But now, because f(x) does not depend on y we can pull f(x) into the integral in y; since p, we can use () to show the latter uantity is bounded by f(x) f(y)dy p p dx. dx f(x) f(y) p dydx. Note that for a given x, the values of y are also restricted to ; so, in fact, if we define h = y x, we have h 2 k 2 and we have the further bound We now sum over all j to find I h 22 k x, x+h I f(x) P k f(x) p dx I h 22 k f(x+h) f(x) p dhdx. x I h f(x+h) f(x) p dhdx. Change the order of integration, take a supremum over all h 2 k 2, and get I f(x) P k f(x) p dx {h h 2 k 2} = 2πω (f,2 k 2) p p. sup f(x+h) f(x) p dx h 2 k 2 x I h Thus f P k f Lp (I) (2π) /p ω (f,2 k 2) p. 5

6 A different argument, which we leave to the reader, is needed when p =. At this point I m supposed to present various properties of ω r (f,t) p, p, like (a) ω r (f,t) p is nondecreasing, (b) ω r (f,t) p 0 as t 0 if p < and f L p (I) or if p = and f is uniformly continuous on I, (c) ω r (f,nt) p n r ω r (f,t) p and ω r (f,λt) p (λ+) r ω r (f,t) p for integer n > 0 and real λ > 0, (d) ω r (f +g,t) p ω r (f,t) p +ω r (g,t) p, (e) ω r (f,t) p = 0 for all t > 0 iff f is a polynomial of total degree < r on I, (f) ω r (f,t) p 2 r f Lp (I). See Chapter 2 of DeVore and Lorentz, Constructive Approximation Theory, for the onedimensional theory. Note that property (d) implies that ω r (,t) p is a semi-norm on L p (I). The only if part of (e) is nontrivial. These properties hold for more general domains than our suare I, for example they hold for functions defined on open, convex Ω. Because of (c) we have, in fact, (4) f P k f Lp (I) Cω (f,2 k ) p. And because of (4) and (b), we have that P k f f in L p (I) as k. Thus, if we define P f = 0 we can write Let m to see that P m f = (P m f P m f)+ +(P f P 0 f) = where the sum converges in L p (I). f = lim m P mf = m (P k f P k f). k=0 (P k f P k f), k=0 Since P k f S k S k and P k f S k, P k f P k f S k. So we can write any f in L p (I) as a convergent sum of elements of S k, k 0. 0 < p < : Not as strange as it might seem If we use P k f(x) = f for x as our projector, then the above stuff works only for p. For certain applications, we will need p < and it will be nice to get it to work there, too. 6

7 (5) The basic ineuality for 0 < p is t i i p i t i p. We can prove this as follows. Without loss of generality we assume that 0 b a and We now note that for x 0, a+b p = a p +(b/a) p = a p (+x) p, 0 x = b/a. satisfies F(0) =, and for 0 < x <, The numerator is F(x) := (+x)p +x p F (x) = (+xp )p(+x) p (+x) p px p (+x p ) 2. p(+x) p [+x p (+x)x p ] = p(+x) p [+x p x p x p ] = p(+x) p [ x p ]. Since p 0 and 0 < x <, this uantity is negative. Thus, F(x) is nonincreasing on [0,] and (+x) p +x p F(0) =, i.e., (+x) p +x p. Backing up gives which in turn gives (5). a+b p = a p (+(b/a)) p a p (+(b/a) p ) = a p +b p, From (5) we can derive other useful ineualities. For example, if 0 < s r < and t i = x i r, then set p = s/r to see that ( xi r ) s/r xi r s/r, or, on taking sth roots of each side, ( xi r ) /r ( xi s ) /s, 7

8 as claimed earlier. i.e., Formula (5) immediately gives ( f(x)+g(x) p dx f(x) p + g(x) p) dx, I (6) f +g p L p (I) f p L p (I) + g p L p (I). The function F(ξ) = ξ p is concave on [0, ) for 0 < p, so for 0 t and any nonnegative a and b tf(a)+( t)f(b) F ( ta+( t)b ). Setting t = /2, a = f Lp (I), and b = g Lp (I), we get with some algebra Combining this with (6) gives f p L p (I) + g p L p (I) 2 p( f Lp (I) + g Lp (I)) p. f +g Lp (I) 2 p ( f Lp (I) + g Lp (I)), i.e., Lp (I) is a uasi-norm. The completeness of L p (I) for p < is proved in the same way as for p <. Projections and moduli of smoothness in L p (I), 0 < p For any nontrivial measurable set Ω we can define a (possibly non-uniue) median of f on Ω to be any number m for which (7) {f(x) m} 2 Ω and {f(x) m} 2 Ω. A median is a best L (Ω) constant approximation to f on Ω. If k < m, for example, then because {f m} 2 Ω f k = (f k)+ (k f) = + = f k f m f<m f<k (f m)+ (m f)+ f m f<m I (m k)+ (k m) k f<m k f<m (f k) (k f) f m +(m k)( {f m} {f < m} )+2 f m. 8 k f<m (f k)

9 We can try a new projector f j,k := P k f := a median of f on. Ij,k Note that this isn t necessarily uniue, so just pick one. We note that {x f(x) f j,k } f j,k p dx f(x) p dx No matter the choice of median, the measure of the set of all x such that f(x) f j,k on is at least 2, so the left hand side is so {x f(x) f j,k } f j,k p = {x f(x) f j,k } f j,k p f j,k p, 2 Summing over all j gives P k f p 2 f p. P k f p L p (I) 2 f p L p (I). or P k f Lp (I) 2 /p f Lp (I). Note that this argument works for any 0 < p <, and the case p = is trivial. Although P k is no longer linear, one sees immediately that P k S = S for S S k and P k is still additive with respect to elements of S k, i.e., one can choose medians so that P k (f +S) = P k f +P k S = P k f +S. So, for p and P k the median operator, we have that P k f Lp (I) 2 f Lp (I) and the same argument as for the averaging operator shows that f P k f Lp (I) 3 inf S S k f S Lp (I). For 0 < p <, use (6) to see that for any S S k, f P k f p L p (I) = f S +S P kf p L p (I) f S p L p (I) + S P kf p L p (I) = f S p L p (I) + P k(s f) p L p (I) 3 f S p L p (I) 9

10 so f P k Lp (I) 3 /p inf S S k f S Lp (I), i.e., P k f is a near-best approximation to f in S k with a constant that depends on p. If we now distinguish P average k f from Pk median we see that for p and k 0, f Pk median f Lp (I) C inf f S Lp (I) C f P average S S k k f Lp (I) Cω (f,2 k ) p. A different argument works for all 0 < p <. For fixed x, we note that for points y that cover at least half the measure of, we have f(x) P k f f(x) f(y), since f(y) and f(x) will lie on opposite sides of the median. So for x and p > 0, f(x) P k f p 2 f(x) P k f p {y Ij,k f(x) f(y) f(x) P k f } = f(x) P k f p dy {y f(x) f(y) f(x) P k f } {y f(x) f(y) f(x) P k f } f(x) f(y) p dy. f(x) f(y) p dy So or 2 f(x) P k f p dx f(x) f(y) p dydx f(x) P k f p dx 2 f(x) f(y) p dydx and we proceed as with the average projector to show that f P k f Lp (I) C(p)ω (f,2 k ) p. What are the properties of the modulus of smoothness when 0 < p? When p, property (d) for the modulus of smoothness is modified to ω r (f +g,t) p p ω r (f,t) p p +ω r (g,t) p p, 0

11 or, for all 0 < p, ω r (f +g,t) min(p,) p ω r (f,t) min(p,) p +ω r (g,t) min(p,) p, Similarly, because r h f(x) p dx = I rh I rh I rh I rh r ( ) r ( ) r+s f(x+sh) s s=0 r ( ) p r f(x+sh) p dx s r ( ) r f(x+sh) p dx s s=0 s=0 2 r f p L p (I) p dx for 0 < p, (f) is now ω r (f,t) min(p,) p 2 r f min(p,) L p (I) for 0 < p. Finally, for integer n > 0 and real λ > 0, (c) is now ω r (f,nt) min(p,) p n r ω r (f,t) min(p,) p and ω r (f,λt) min(p,) p (λ+) r ω(f,t) min(p,) p. We can choose many other nonlinear projectors. One that I find interesting is ( ) P k f = round f where round(x) is the integer closest to x (breaking ties arbitrarily). It is shown in DeVore, Jawerth, Lucier Image compression through wavelet transform coding that if f takes on integer values (as it does from a digital camera, for example), then this P k f is stable for all 0 < p. This leads to so-called integer-to-integer wavelet transforms (which necessarily are nonlinear) and wavelet transforms for binary images. One can even calculate the averages using fixed-point arithmetic with enough precision and still have a stable transform. This nonlinear theory can accomodate many computational crimes.

12 Besov smoothness spaces We measure the smoothness of f by considering how fast the modulus of smoothness decays as t 0. One natural measure may be: ( C, > 0) ( t > 0), This is euivalent to and, in fact, we could take ω r (f,t) p Ct. t ω r (f,t) p C supt ω r (f,t) p t as a semi-norm if p (and a semi-uasi-norm if 0 < p < ). Unfortunately, this is not uite general enough for our purposes. We need to make a more subtle distinction in the decay of ω r (f,t) p, so we add another parameter and define for 0 < p, and r < r the (uasi-)semi-norms for the Besov spaces B (L p (I)) by ( f B (L p (I)) := [t ω r (f,t) p ] dt ) /, 0 < <, t and We also define the (uasi-)norm 0 f B (L p (I)) := sup t ω r (f,t) p. 0<t< f B (L p (I)) := f B (L p (I)) + f Lp (I). (And after that bit of pedantic distinction between(uasi-)(semi-)norms and(semi-)norms, we shall in the future call uasi-norms norms. ) Now and we have for 2 k t 2 k 0 [t ω r (f,t) p ] dt t = 2 k [t ω r (f,t) p ] dt 2 t. k [2 k ω r (f,2 (k+) ) p ] 2 k [t ω r (f,t) p ] t [2(k+) ω r (f,2 k ) p ] 2 k+ We integrate over [2 (k+),2 k ] (which has length 2 (k+) ), sum over k, and use property (c) for moduli of smoothness (arguing separately for p and 0 < p < ) to see that there are positive constants c and c such that ( ) / ( ) / c [2 k ω r (f,2 k ) p ] f B (L p (I)) c [2 k ω r (f,2 k ) p ], 2

13 i.e., f B (L p (I)) {2 k ω r (f,2 k ) p } l. So we can take as the norm of B (L p(i)) the uantity f B (L p (I)) := f Lp (I) + {2 k ω r (f,2 k ) p } l. Seuence norms of wavelet coefficients: Part I, the direct ineuality We have decomposed f L p (I) by f = (P k f P k f), where we can take P k to be the average projector onto S k when p and we take P k to be the median projector onto S k for any p > 0. Because P k f S k S k, we have P k f P k f S k so we define d j,k by P k f P k f = d j,k χ j,k ; j Z 2 k note that ( /p P k f P k f Lp (I) = d j,k χ j,k p L p (I)) = { d j,k χ j,k Lp (I)} lp (j Z 2 k ). j Now for 0 < p and k we have P k f P k f Lp (I) = P k f f +f P k f Lp (I) C( P k f f Lp (I) + P k f f Lp (I)) C(ω (f,2 k ) p +ω (f,2 (k+) ) p ) Cω (f,2 k ) p, while for k = 0, P 0 f P f Lp (I) = P 0 f Lp (I) = d 0,0 C f Lp (I). 3

14 Combining these bounds, we see that for any 0 < we have {2 k P k f P k f Lp (I)} l () = {2 k { d j,k χ j,k Lp (I)} lp (j Z 2 k ) } l () C( {2 k ω (f,2 k ) p } l () + f Lp (I)) = C f B (L p (I)). We tend to keep χ j,k Lp (I) in there as a weight; we can also write the left hand-side of the ineuality as {2 k χ j,k Lp (I) {d j,k } lp (j Z 2 k ) } l (). Since we have the explicit value χ j,k Lp (I) = 2 2k/p we can also write the left-hand-side of the ineuality as {2 ( 2/p)k {d j,k } lp (j Z 2 k ) } l (). If we use a different normalizationfor χ j,k then the weight 2 ( 2/p)k willdiffer; normalizing χ j,k to be have χ j,k Lp (I) = rather than just being the characteristic function of, for example, will lead to a weight of 2 k. The direct ineuality (8) {2 k χ j,k Lp (I) {d j,k } lp (j Z 2 k ) } l () C f B (L p (I)), which followed from the Jackson ineuality f P k f Lp (I) Cω (f,2 k ) p, is the easy part. Seuence norms of wavelet coefficients: Part II, the inverse ineuality The harder part is to show that f B (L p (I)) C {2 k P k f P k f Lp (I)} l (), which is called the inverse ineuality. The inverse ineuality will follow by a bound on the modulus of smoothness of any element of S k ; specifically for S S k we have the Bernstein ineuality (9) ω (S,t) p C { S Lp (I), 2 k t, 2 k/p t /p S Lp (I), 0 < t 2 k. 4

15 The first ineuality is a special case of To prove the second, we start with ω r (S,t) min(,p) p 2 r S min(,p) L p (I). f(x+h) f(x) = f(x+(h,h 2 )) f(x+(h,0))+f(x+(h,0)) f(x), h = (h,h 2 ), so hf Lp (I h ) C p ( (h,0) f L p (I (h,0)) + (0,h 2 ) f L p (I (0,h2 ))), so we need to consider only offsets parallel to the coordinate axes. Now we denote S = s j,k χ j,k. j When x and 0 h 2 k, we have { sj,k s j,k = 0, x+(h,0), S(x+(h,0)) S(x) = s j+(,0),k s j,k, x+(h,0) I j+(,0),k. The area where x and x+(h,0) I j+(,0),k is h 2 k ; therefore S(x+(h,0)) S(x) p dx = S(x+(h,0)) S(x) p dx I (h,0) j I (h,0) = s j+(,0),k s j,k p h2 k j +<2 k = 2 k h s j+(,0),k s j,k p 2 2k j +<2 k This implies the second part of (9). = 2 k h (2 k,0) S p L p (I (2 k,0) ) C2 k h S p L p (I). If we write S k = P k f P k f S k, we have f = S k. If 0 < p we have hf(x) p = I h I h p hs k hs k p. I h 5

16 If we now take h 2 m and use the two parts of (9), we find ( h f p C h S k p + ) h S k p I h 0 k<m I h m k I h ( C 0 k<m(2 k 2 m ) S k p L p (I) + ) S k p L p (I). m k Taking suprema over h 2 m gives ω (f,2 m ) p p C ( and multiplying by 2 mp gives 0 k<m2 k m S k p L p (I) + m k S k p L p (I) ), m 0, ( 2 mp ω (f,2 m ) p p C 0 k<m2 k+(p )m S k p L p (I) + ) 2 mp S k p L p (I). m k Now we see that we want to have 2 kp S k p L p (I) on the right hand side, so we multiply and divide each term on the right by 2 kp : 2 mp ω (f,2 m ) p p C ( 0 k<m2 (p )(m k) 2 kp S k p L p (I) + m k This set of ineualities can be written in vector form. We set Then, componentwise in m, x m := 2 mp ω (f,2 m ) p p and y k := 2 kp S k p L p (I). 2 (m k)p 2 kp S k p L p (I) 2 p 2 2p 2 3p... 2 (p ) 2 p 2 2p... (x m ) C 2 2(p ) 2 (p ) 2 p... (y k ). 2 3(p ) 2 2(p ) 2 (p )..... If p we proceed similarly, but now we use the triangle ineuality and end up with ( 2 m ω (f,2 m ) p C 0 k<m2 ( /p)(m k) 2 k S k Lp (I) + m k (m k) 2 k S k Lp (I) ) ).

17 Now we set and get the vector ineuality x m := 2 m ω (f,2 m ) p and y k := 2 k S k Lp (I) ( /p) (x m ) C 2 2( /p) 2 ( /p) 2... (y k ). 2 3( /p) 2 2( /p) 2 ( /p) These two vector ineualities, which are the crux of the matter, were derived by Larry Brown at Purdue. In either case for l 0 we set T l (y 0,y,y 2,...) := (y l,y l+,...) and T l (y 0,y,y 2,...) := (0,...,0,y }{{} 0,y,...). l times For any 0 < r, we obviously have T l (y k ) r (y k ) r. For 0 < p our vector ineuality can be written We set r = /p; if 0 < r, we have ( ) (x m ) C 0 l2 lp T l (y k )+ 2 l(p ) T l (y k ). l ( ) (x m ) r r C 0 l2 lpr T l (y k ) r r + 2 l(p )r T l (y k ) r r. l Both these sums are finite if p < 0, i.e., < /p, and we get (x m ) r r = m 0 [2 mp ω (f,2 m ) p p ]/p C (y k ) r r = C [2 kp S k p L p (I) ]/p which is what we want upon taking th roots. If r, then we have ( (x m ) r C 0 l2 lp T l (y k ) r + 2 l(p ) T l (y k ) r ); l again, both these sums are finite if < /p, and (x m ) r C (y k ) r, 7

18 which again gives what we want upon taking pth roots. For p our vector ineuality becomes ( ) (x m ) C 0 l2 l T l (y k )+ 2 l( /p) T l (y k ). l Now we take r = ; again, whether 0 < r or r, we have for < /p In a similar way we can show m 0[2 m ω (f,2 m ) p ] C [2 k S k Lp (I)]. (0) f Lp (I) C {2 k S k Lp (I)} l (0 k). So we have ( ) / () f B (L p (I)) C [2 k P k f P k f Lp (I)], and from (8) and () we have proved the following theorem: Indeed, if p then f Lp (I) S k Lp (I); if we have ( / ( S k Lp (I) S k L p (I)) 2 k S k L p (I) and if we have (with / +/ = ) S k Lp (I) = ( S k Lp (I)2 k 2 k S k L p (I) 2k)/( ) / 2 k ) /, so we ve proved (0) for p. If p then f p L p (I) S k p ; if r = /p then L p (I) S k p L p (I) ( ) /r ( p/ ( p/ S k pr = S L p (I) k L p (I)) 2 k S k L p (I)), which implies (0), and if r we have (again with /r +/r = ) k S p L p (I) = ( ) ( /r S k p L p (I) 2kp 2 kp S k pr ) /r L p (I) 2kpr 2, kpr which, since pr =, again implies (0). 8

19 Theorem. Assume 0 < p, 0 < < min(,/p), 0 <. For x we let f, p, P k f(x) = a median of f on, 0 < p. Then (2) f B (L p (I)) {2 k P k f P k f Lp (I)} l (0 k). Embeddings of Besov spaces One can derive a number of properties of Besov spaces from Theorem. Because L p (I) L p (I) when p p, we have in that case B (L p(i)) B (L p (I)). Because l l if 0 < <, we have B (L p (I)) B (L p(i)) for 0 < <. In particular, B (L p (I)) B (L p (I)) for all 0 < <. If >, however, then B (L p(i)) B(L p (I)) for any and. One sees this by noting that B (L p(i)) B (L p(i)) and f B (L p(i)) implies there is a C such that This means that for all k 2 k P k f P k f Lp (I) C. 2 k P k f P k f Lp (I) C2 ( )k, and the right-hand side, being a decreasing geometric seuence, is in l for any. Thus is the main determiner of smoothness for fixed p, and the second parameter allows us to make finer distinctions. (3) We now fix δ R and consider the pairs, ( > 0, 0 < < min(,/)) that satisfy 2 = δ. For any pair satisfying (3) we have with P k f P k f = j Z d 2 j,k χ j,k, k {2 k P k f P k f L (I)} l (0 k) = 2 k d j,k χ j,k L (I) j Z 2 k = 2 k d j,k 2 2k = j Z 2 k 2 ( 2/)k d j,k = j Z 2 k 2 2kδ d j,k. 9 j Z 2 k

20 So (4) f B (L (I)) {2 k P k f P k f L (I)} l (0 k) = {2 2kδ d j,k } l (, j Z 2 k ). If the pair, also satisfies (3) with >, then < ; because l l in this case, we have that B (L (I)) B (L (I)). All these inclusions come with norm ineualities, so they are in fact embeddings. To summarize: for, > 0, 0 < p,p,,, B (L p (I)) B (L p(i)) when p > p; B (L p(i)) B (L p (I)) when < ; B (L p(i)) B (L p(i)) when > for any, ; B (L (I)) B (L (I)) when 2 = 2 and >. In fact, we have for 0 <, 0 < p <, and = 2 + p ( so δ = p that B(L (I)) L p (I). This can be shown simply when p = 2, so = 2 + ( so δ = ), 2 2 P k f is the best L 2 (I) approximation to f on S k, and 0 < <. For then P k f P k f is orthogonal to all S S l for l k, and for fixed k the χ j,k, j Z 2 k, are orthogonal because they have essentially disjoint support. Thus we have ), (5) f 2 L 2 (I) = P k f P k f 2 L 2 (I) = d j,k χ j,k 2 L 2 (I) = 2 k d j,k 2 = j Z 2 k 2 2kδ d j,k 2. j Z 2 k = {2 2kδ d j,k } l2 (, j Z 2 k ). j Z 2 k Again, because < 2, l l 2. Thus from (5) and (4) we have, as claimed, B (L (I)) L 2 (I). We summarize these results in the important Figure. A function space with smoothness in L p (I) is graphed at the point (/p,). This concept is not so precise, so the Sobolev space W, (I), the Besov space B (L (I)) for any 0 <, and the space of functions of bounded variation BV(I) would all be represented by the point (, ). Similarly, all Besov spaces B (L p(i)) for any would be graphed at the point (/p,), and the space L p (I) would be represented by the point (/p,0) (since functions in L p (I) have zero smoothness). 20

21 B (L p (I)) p 2 = δ 0 δ p (I)) is embedded in every Besov space Figure. The Besov space B (L p B (L p(i)) with (/p,) in at least the (open) shaded area. Two-dimensional Haar transform This may not look much like regular wavelets, so we specialize a bit further and see how this applies to two-dimensional Haar wavelets. For each k > 0, each interval = I 2j,k I 2j+(,0),k I 2j+(0,),k I 2j+(,),k, so naturally we can write on P k f P k f = d 2j,k χ 2j,k +d 2j+(,0),k χ 2j+(,0),k +d 2j+(0,),k χ 2j+(0,),k +d 2j+(,),k χ 2j+(,),k = d 2j+(0,),k d 2j+(,),k d 2j,k d 2j+(,0),k, where we have boxed the values of the difference of projections to suggest the values in each uarter of the interval We choose a different basis for this four-dimensional space: For any numbers a, b, c, and d, we can write (6) a b c d = + + +β + + +γ + + +δ Therefore, for any projector P k we can write P k f P k f as a linear combination P k f P k f = c j,k,ψ ψ j,k j Z 2 k 2 ψ Ψ

22 where Ψ = {ψ (),ψ (2),ψ (3),ψ (4) } and each ψ is defined on I by So we can write {, 0 x < /2, ψ () (x,x 2 ) =, otherwise, {, 0 x2 < /2, ψ (2) (x,x 2 )) =, otherwise, { 0 x,x ψ (3) 2 < /2 or /2 x,x 2, (x,x 2 ) =, otherwise, ψ (4) (x,x 2 ) = for all (x,x 2 ) I. (7) f = (8) j Z 2 ψ Ψ k c j,k,ψ ψ j,k. On each interval, we have for any 0 < p < ( ) p d l,k χ l,k ψ l c j,k,ψ ψ j,k p, with constants that depend only on p because all (uasi-)norms on a four-dimensional space are euivalent. Thus, from (2) and (8) we have the norm euivalence f B (L p (I)) ( When P k is the average projector, [ ( /p ] ) / 2 k c j,k,ψ ψ j,k p L p (I)). j,ψ P k f(x) = f = f j,k for x, we can specialize things a bit more. We have on f j,k = 4 (f 2j,k +f 2j+(,0),k +f 2j+(0,),k +f 2j+(,),k ), i.e., f j,k is the average of the four k-level pixel values on, and the coefficient δ in (6) is zero. (When k = 0, δ is the average value of f on I.) 22

23 We then write Ψ 0 = Ψ and and normalize ψ j,k in L 2 (I): Ψ k = {ψ (),ψ (2),ψ (3) }, k > 0, ψ j,k (x) = 2 k ψ(2 k x j), k 0, ψ Ψ k,j Z 2 k. The set {ψ j,k k 0, ψ Ψ k, j Z 2 k } is orthonormal and, because of (7), forms an orthonormal basis for L 2 (I). In particular, if f = c j,k,ψ ψ j,k (the Haar transform), then ψ Ψ k j Z 2 k (9) f L2 (I) = (c j,k,ψ ) l2 (0 k,j Z 2 k,ψ Ψ). Furthermore, since ( ) /p ψ j,k Lp (I) = [2 k ] p = (2 2k 2 kp ) /p = 2 k( 2/p) for all ψ Ψ k and j Z 2 we can write for 0 < < p and p ( [ ( f B (L p (I)) 2 k 2 k( 2/p) ψ Ψ k j Z 2 k because the average projector P k is bounded on L p (I). In particular, if 0 < < and p = satisfies = 2 + 2, ) /p ] ) / c j,k,ψ p so < < 2, then the exponent of 2 in the previous formula satisfies ( + 2 ) k = 0, p and (20) f B (L (I)) (c j,k,ψ ) l (0 k,j Z 2 k,ψ Ψ). Since l l 2, we see immediately from (9) and (20) that B (L (I)) is embedded in L 2 (I). In the following we accept implicitly for the Haar transform that ψ (4) will be used only when k = k,

24 The big picture, part I: Linear approximation In the next two sections we discuss two big ideas in the context of wavelets: () Nonlinear approximation is better than linear approximation. (2) Approximation is euivalent to smoothness. We ask a number of uestions: () What do we mean by linear and nonlinear approximation by wavelets? (2) What does it mean to approximate a function well by wavelets? (3) Can one characterize the set of functions that are approximated well by wavelets? We note that the average projector P k f is the best approximation in L 2 (I) to f on S k, and using the Haar wavelets we developed in the previous section we can write (2) P k f = 0 l<k, j Z 2 l, ψ Ψ c j,l,ψ ψ j,l, c j,l,ψ = f,ψ j,l. There are 2 2k terms in the sum in (2), and the dimension of S k is 2 2k. So for each k we have chosen, a priori, a set of 2 2k wavelet terms {ψ j,l 0 l < k, j Z 2 l, ψ Ψ} before even looking at f. We also have that P k (f +βg) = P k f +βp k g, i.e., this approximation process is linear. So if we define for p E N (f) p = inf S S k f S Lp (I), N = 2 2k (the dimension of S k ), the error of best approximation of f in L p (I), we have by (3) and (4), E 2 2k(f) p f P k f Lp (I) 2E 2 2k(f) p Cω (f,2 k ) p. So, for any p <, 0 < < /p, and 0 <, {2 k E 2 2k(f) p } l () C {2 k ω (f,2 k ) p } l () C f B (L p (I)). 24

25 Conversely, from Theorem, f B (L p (I)) {2 k P k f P k f Lp (I)} l () {2 k ( P k f f Lp (I) + f P k f Lp (I))} l () C {2 k P k f f Lp (I)} l () C {2 k E 2 2k(f) p } l (). Combining the previous two ineuality gives {2 k E 2 2k(f) p } l () f B (L p (I)), i.e., f can be approximated in S k at a certain rate if and only if f is in a specific Besov smoothness space. This is the first example of the dictum: Approximation is euivalent to smoothness. The big picture, part II: Compression of wavelet coefficients Note: This section is not yet written in a way that fits in with the previous material. We choose an error space; for the moment, we choose L 2 (I). We also work with orthonormal Haar wavelets {ψ j,k }. We note that if we want to approximate f = c j,k,ψ ψ j,k j,k,ψ by a sum f = c j,k,ψ ψ j,k c j,k,ψ ψ j,k Λ with no more than N terms in Λ and we want to minimize ( f f L2 (I) = c j,k,ψ ψ j,k Λ c j,k,ψ 2 ) /2 then we should put into Λ the N terms c j,k,ψ ψ j,k with the largest values of c j,k,ψ ; we call that approximation f N. (Break ties in an arbitrary manner.) If we sort {c j,k,ψ ψ j,k } in nonincreasing order of c j,k,ψ, and call the resulting seuence {c i } = { c j,k,ψ }, c i c i+, 25

26 then ( /2 f f N L2 (I) = ci) 2. N<i We now want to find an euivalence between the rate of decay of f f N L2 (I) as N and the smoothness of f in B (L (I)), if in fact ( ) / ( / f B (L (I)) c j,k,ψ = ci) <. j,k,ψ i A simple bound on f f N L2 (I) when f B (L (I)), / = /2 + /2, can be obtained as follows. We choose an ǫ > 0 and ask How many coefficients can satisfy c j,k,ψ > ǫ? If we denote this number by N, then we must have ( (Nǫ ) / c j,k,ψ >ǫ ) / ( ) / c j,k,ψ c j,k,ψ C f B (L (I)), j,k,ψ so N Cǫ f B (L (I)) and ǫ < CN / f B (L (I)). We put these N coefficients into Λ, the resulting error is ( f f N L2 (I) = c i ǫ /2 ( ) /2 ci) 2 supc (2 )/2 i c i c i ǫ c i ǫ ǫ (2 )/2 ( c i ) /2 C(N / f B (L (I))) (2 )/2 f /2 B (L (I)) = CN (2 )/(2) f B (L (I)) = CN /2 f B (L (I)) since 2 = 2. We can rewrite this as N /2 f f N L2 (I) C f B (L (I)), 26

27 or (22) {N /2 f f N L2 (I)} C f B (L (I)). In fact, we have the more subtle estimate (23) ( [N /2 f f N L2 (I)] N) / ( [ ( = N /2 N N i= i=n c 2 i ) /2 ] ) / N ( / ci) f B (L (I)), which we prove using a lemma from DeVore and Temlyakov, Advances in Computational Math, Vol 5, 996, I m responsible for any errors in translation. Note that (23) implies (22) since N /N diverges. Let {a k} be a non-negative, non-increasing seuence, and Then we have So or In the other direction, ( σ 2 m = k=2 m a 2 k a 2m a 2m a m 2 m= ( ) /2 m= a m m= σm 2 = a 2 k, k=m = ( 2m m k=m a 2 k m /2σ m = 2 ) /2 m /2σ m. m /2 σm m m= ) / ( 2 / [m /2 σ m ] m) /. m= ( ) /2 2 k a 2 2 since a k k a 2 m for the 2 m terms k = 2 m,...,2 m+ k=m ( ) / 2 k/2 a 2 k k=m 27

28 since < 2. Thus, 2 m/2 σ 2 2 m/2 2 k/2 a m 2 k m=0 = = m=0 k=m k 2 k/2 2 m/2 a (change order of summation) 2 k k=0 k=0 2/2 2 /2 = 2/2 2 /2 2/2+ 2 /2 m=0 2 k/22(k+)/2 a 2 /2 2 k 2 k/2 2 k/2 a 2 k k=0 (geometric series) 2 k a, since /2+/2 =, 2 k k=0 j= since a 2 k a j for the 2 k terms with j = 2 k +,...,2 k. We also have a j σ 2 m σ j σ 2 m+ and 2 m/2 j /2 < 2 (m+)/2 for the 2 m terms with 2 m j 2 m+. So 2 m+ j=2 m j /2 σ j Combining these ineualities gives us [j /2 σ j ] j 2/2 j= which is what we needed. j 2(m+) σ 2 m 2 m2m 2 /2 2 m/2 σ 2 m. m=0 2 m/2 σ 2 m 2+ 2 /2 a j, j= What is a wavelet? In the course of presenting these notes I was asked What is a wavelet?. There are many people who can give a better answer than I can, but I ll give my perspective. 28

29 I prefer the uestion What is a wavelet transform? The reader should realize that the rest of this section is meta-mathematics, not mathematics. Let φ: R 2 R be a refinable function, that is, there are finitely many coefficients a j, j Z 2, for which φ(x) = a j φ(2x j). j For each k 0, let S k be the span of the functions φ jk (x) = φ(2 k x j), j Z 2, whose support intersects with the unit interval I nontrivially. Assume that there are (possibly nonlinear) projectors P k : L p (I) S k for some range of p and a positive constant C such that for some r > 0 and all k 0 and all f L p (I), (24) P k f f Lp (I) Cω r (f,2 k ) p. Assume also that there is some β > 0 and C > 0 such that for all S S k, (25) ω r (S,t) p C { S Lp (I), 2 k t, 2 kβ/p t β/p S Lp (I), 0 < t 2 k. In other words, Jackson (24) and Bernstein (25) ineualities hold for the spaces S k. In this case, the series f = (P k f P k f) converges in L p (I) and Theorem will hold for some range of and p. Conclusion: This combination of a family of subspaces S k, generated by the dyadic dilatesandtranslatesofafunctionφ, togetherwithspecificprojectorsp k suchthatjackson (24) and Bernstein (25) ineualities hold is a wavelet transform. In most cases people construct linear projectors P k such that that S k is the direct sum of S k and the range of P k P k, which we ll denote by W k. So S k = S k W k. Because φ is refinable, the space W k is generated by 2 d functions ψ Ψ, where we re working in R d (d = 2 up until now). If the functions φ( j), j Z 2, are mutually orthogonal, then the functions ψ( j), j Z 2, ψ Ψ can be taken to be mutually orthogonal. This is the situation for the Haar transform, for which φ = χ I, the characteristic function of I, and P k is the L 2 (I) projection onto S k. 29

30 If P k is the median transform, however, then it s not hard to see that the range of P k P k is S k itself. Nonetheless, with this specific way of calculating coefficients, Jackson and Bernstein ineualities hold, and we consider this a wavelet transform. This definition is at the very least imprecise and most likely incorrect in important ways. But it is general enough to allow the notion of nonlinear wavelet transforms, binary wavelet transforms (apply median projectors to functions f whose ranges take only the values 0 and ), integer-to-integer wavelet transforms (see Devore-Jawerth-Lucier, Image compression through wavelet transform coding ), etc. 30