Near best tree approximation

Transcription

1 Advances in Computational Mathematics 16: , Kluwer Academic Publishers. Printed in the Netherlands. Near best tree approximation R.G. Baraniuk a,,r.a.devore b,g.kyriazis c and X.M. Yu d a Department of Electrical and Computer Engineering, Rice University, Houston, TX 77005, USA richb@rice.edu b Department of Mathematics, University of South Carolina, Columbia, SC 29208, USA devore@math.sc.edu c Department of Mathematics and Statistics, University of Cyprus, Nicosia, Cyprus kyriazis@ucy.ac.cy d Department of Mathematics, Southwest Missouri State University, Springfield, MO 65804, USA xmy944f@smsu.edu Received 5 December 2000; accepted 12 October 2001 Communicated by Y. Xu Tree approximation is a form of nonlinear wavelet approximation that appears naturally in applications such as image compression and entropy encoding. The distinction between tree approximation and the more familiar n-term wavelet approximation is that the wavelets appearing in the approximant are required to align themselves in a certain connected tree structure. This makes their positions easy to encode. Previous work [4,6] has established upper bounds for the error of tree approximation for certain (Besov) classes of functions. This paper, in contrast, studies tree approximation of individual functions with the aim of characterizing those functions with a prescribed approximation error. We accomplish this in the case that the approximation error is measured in L 2, or in the case p 2, in the Besov spaces Bp 0(L p), which are close to (but not the same as) L p. Our characterization of functions with a prescribed approximation order in these cases is given in terms of a certain maximal function applied to the wavelet coefficients. Keywords: compression, n-term approximation, encoding, approximation classes AMS subject classification: 41A25, 41A46, 65F99, 65N12, 65N55 1. ntroduction Tree approximation is a form of nonlinear wavelet approximation that occurs in the application of wavelets to image processing and adaptive methods. The usual nonlinear wavelet approximation, known as n-term approximation, seeks to approximate a target function f by a linear combination of at most n wavelets. The approximation properties This work has been supported in part by Office of Naval Research contracts N and N , National Science Foundation grants MP and CCR , and Army Research Office contract DAAG Corresponding author.

2 358 R.G. Baraniuk et al. / Near best tree approximation of n-term wavelet approximation are by now well-known [7,10]. n particular, there is a characterization of the functions that can be approximated with a prescribed rate of approximation in terms of the wavelet coefficients of the target function (and this can sometimes be restated directly in terms of the smoothness of the target function). One deficiency in n-term approximation, which manifests itself in computations, is that the terms appearing in an n-term approximant can involve any wavelets. Thus, for example, to encode an n-term approximant, one has to not only to assign bits for the coefficients of the decomposition but also to indicate which particular wavelets appear in the decomposition. The cost of the latter can be prohibitive. Tree approximation is designed to overcome this objection. Wavelets are naturally indexed on dyadic cubes. Dyadic cubes have a tree structure under inclusion. Tree approximation differs from n-term approximation in requiring that the wavelet functions in the approximant should correspond to a subtree of the indexing tree. This allows for efficient encoding of the positions of the wavelets used in the approximation. Previous work [4,6] has given upper bounds for the efficiency of tree approximation on certain smoothness classes and has shown the optimality of these bounds in the sense of Kolmogorov entropy. The purpose of this paper is to go further and actually characterize the functions with a prescribed rate of tree approximation. The characterization is given in terms of a certain maximal function applied to the wavelet coefficients of the target function. Section 2 introduces the wavelet setting in which we will work and states the main results of the paper. Section 3 investigates several key properties required to prove our best tree approximation results in sections 4 and Main results n order to describe tree approximation and the results of this paper, we begin by recalling the usual setting for wavelet decompositions and nonlinear approximation. The results of this paper can be established for quite general wavelet decompositions, but we shall restrict ourselves to the compactly supported biorthogonal wavelets of Cohen et al. [5]. These include, as a special case, the orthogonal wavelets of compact support of Daubechies [8]. A good reference for these bases and their properties is the monograph of Daubechies [9, chapter 8] Wavelets The construction of biorthogonal wavelets begins with two compactly supported univariate scaling functions φ and φ whose shifts are in duality: φ(x k) φ ( x k ) dx = δ ( k k ), k,k Z, (2.1) R

3 R.G. Baraniuk et al. / Near best tree approximation 359 with δ the Kronecker delta and the overline denoting complex conjugation. Associated to each of the scaling functions are mother wavelets ψ and ψ. These functions can be used to generate a wavelet basis for the L p (R d ) spaces as follows. For any function g defined on R d and any dyadic cube = 2 j [k, k + 1], k Z d, we define the function g (x) := 2 jd/2 g ( 2 j (x k) ) (2.2) which is a rescaling of g to normalized for L 2 (R d ).Wedefineψ 0 := φ, ψ 1 := ψ. Let V denote the collection of vertices of the unit cube [0, 1] d and V the nonzero vertices. For each vertex v = (v 1,...,v d ) V, we define the multivariate functions ψ v (x 1,...,x d ) := ψ v 1 (x 1 ) ψ v d (x d ), ψ v (x 1,...,x d ) := ψ v 1 (x 1 ) ψ v d (x d ). (2.3) Let D denote the set of all dyadic cubes in R d and D j thecubesind that satisfy = 2 jd,where E denotes the Euclidean measure of a set E. The collection of functions ψ v, D, v V (2.4) are a Riesz basis for L 2 (R d ) (in the orthogonal case they form a complete orthonormal basis for L 2 (R d )). They are an unconditional basis for L p (R d ),1<p<. Each function f that is locally integrable on R d has the wavelet expansion f = a v (f )ψ v, av (f ) := f, ψ v. (2.5) D v V We can start the wavelet decomposition at any dyadic level. For example, starting at dyadic level 0, we obtain f = a v (f )ψ v + a v (f )ψ v. (2.6) v V j=1 v V D 0 t is sometimes convenient to choose different normalizations for the wavelets and coefficients appearing in the decompositions (2.5), (2.6). n (2.5), (2.6), we have normalized in L 2 (R d ); we can also normalize in L p (R d ),0<p,bytaking Then, we can rewrite (2.6) as f = where with 1/p + 1/p = 1. D j ψ v,p := 1/p+1/2 ψ v, D, v V. (2.7) D 0 v V a v,p (f )ψ v,p + a,p v (f )ψ,p v, (2.8) j=1 D j v V a v,p (f ) := f, ψ v,p (2.9)

4 360 R.G. Baraniuk et al. / Near best tree approximation For simplicity of notation, we shall combine all terms associated to a dyadic cube in the following a,p v (f )ψ,p v, D 0, v V A (f ) := a,p v (f )ψ,p v, D (2.10) j,j 1. v V Note that the definition of A (f ) does not depend on p and that ( a v A (f ),p (f ) ) 1/p p, D 0, Lp (R d ) a v V,p(f ) := ( a v,p (f ) ) 1/p p, D j,j 1. v V (2.11) Here and later in this paper a statement A B means that A/B is bounded from above and below by positive constants which do not depend on the variables involved. t is easy to go from one normalization to another. For example, for any 0 < p, q, wehave ψ,p = 1/q 1/p ψ,q, a,p (f ) = 1/p 1/q a,q (f ). (2.12) With this notation, we can rewrite the wavelet decomposition (2.6) as f = D + A (f ) (2.13) with D + := j=0 D j Nonlinear approximation n numerical applications of wavelets, we seek efficient approximations to f that use only a small number of terms from its wavelet decomposition. One way of accomplishing this is through what is known as n-term approximation. We shall consider in this paper only approximation on the whole of R d. However, results can be obtained for more general domains as described for example in [4]. We first consider the approximation of functions from L p := L p (R d ).Let n be defined as the set of all functions S = A (S), # n, (2.14) where the sets are subsets of D +.Givenf L p ( ),1 p, wedefine σ n (f ) p := inf f S Lp, S n n = 0, 1,... (2.15) Note that by definition σ 0 (f ) p := f Lp.

5 R.G. Baraniuk et al. / Near best tree approximation 361 One of the main accomplishments in nonlinear approximation has been the characterization of those functions f for which σ n (f ) p has a prescribed asymptotic behavior as n.for1 p,0<q,ands>0, we define the approximation class A s q (L p( )) to be the set of all f L p ( ) such that ( ) [ ] (n + 1) s q 1 1/q σ n (f ) p, 0 <q<, f A s q (L p ( )) := n + 1 n=0 (n + 1) s σ n (f ) p, q =, sup n 0 (2.16) is finite. From the monotonicity of σ n (f ) p, it follows that (2.16) is equivalent to ( ) [ ] 1/q 2 js q σ 2 j (f ) p, 0 <q<, f A s q (L p ( )) j 1 sup 2 js σ 2 j (f ) p, q =, j 1 (2.17) where for the purposes of this formula σ 1/2 (f ) p := σ 0 (f ) p. Note that in the special case that q =, A α (L p( )) consists of the functions f for which σ n (f ) p = O(n α ). t is possible to characterize the spaces A s q (L p( )) in several ways: in terms of interpolation spaces; in terms of wavelet coefficients; and in terms of smoothness spaces (Besov spaces). See [7] for a discussion of these characterizations. Here, we will just state the characterization in terms of wavelet coefficients that will serve as a comparison for the results we obtain concerning tree approximation. For this we recall the Lorentz spaces l τ,q. Given a sequence (a ) D+,let(αn ) n 1 denote its decreasing rearrangement. That is, αn is the nth largest of the a. Then, l τ,q consists of all sequences (a ) D such that (a ) lτ,q D := n 1/τ αn lq (w), (2.18) is finite, where w is the Haar measure on the positive integers (w(n) := 1/n, n 1). Theorem 2.1 [7,10]. Let 1 <p< and let ψ, ψ be a biorthogonal wavelet pair. For each α > 0, we have that a function f is in A α q (L p( )) if and only if the sequence (a,p (f )) D+ ( ) defined by (2.11) is in the Lorentz sequence space l τ,q with 1/τ := α + 1/p and f A α q (L p ( )) f Lp ( ) + ( a,p (f ) ) D + ( ) lτ,q. (2.19) Results similar to (2.19) hold in the case 0 <p 1 by replacing L p by the Hardy space H p.also,whenq = τ, the space A α τ (L τ ) can be characterized as a Besov space for a certain range of α that depends on the smoothness of ψ and the number of vanishing moments of ψ.

6 362 R.G. Baraniuk et al. / Near best tree approximation 2.3. Wavelet trees As noted earlier, one of the deficiencies in n-term approximation is that the terms appearing in an n-term approximant can occur in any position. The idea in tree approximation is to circumvent this difficulty by requiring that these terms be organized in a tree structure. We introduce some of the notation and terminology for trees that we will use throughout the paper. A tree T of dyadic cubes from D + consists of a collection of cubes with a special nesting property: f < 1and T, then its parent (the smallest dyadic cube containing ) isalsoint. We will refer to the cubes in a tree as its nodes, with #T the number of nodes. A subtree T is a collection of cubes from D + such that whenever and are in T, then any cube K satisfying K is also in T. The cubes T with =1 (that are not contained in any other cubes from T )are called its roots. We will use the notation T to denote a subtree with a single root. Each d-dimensional dyadic cube = 2 j [k, k + 1], k Z d,atlevelj 0 equals the union of 2 d child cubes at level j + 1. f T is a subtree, then we say that the cube D + is a child of T if is not in T but is the child of some cube from T. We shall frequently make use of the following remark. Remark 2.2. Given two subtrees T and T 0 T with common root, we have T \ T 0 = T, (2.20) where is a collection of children of T Tree approximation We denote by n t the collection of all functions S = A (S), #T n, (2.21) T with T a tree. Given a quasi-normed space X, we define the error of tree approximation by t n (f ) X := inf f S X. (2.22) S n t The main point of this is to obtain a characterization for tree approximation similar to theorem 2.1. This will be accomplished by replacing the sequence of wavelet coefficients by a related sequence (b (f )) obtained by applying a certain maximal function to the sequence (a,p (f )) of wavelet coefficients. We shall obtain such characterizations when the approximation error is measured in L 2 or when it is measured in the Besov space B p as defined below. This Besov space is very close to (but different from) L p. There remains the problem of characterizing the approximation classes when the approximation error is measured in L p, p 2.

7 R.G. Baraniuk et al. / Near best tree approximation 363 For 0 <p<, we define the space B p := Bp 0 to consist of all functions f that are locally integrable on and satisfy f Bp := ( a,p (f ) ) ( ) 1/p lp (D + ) = a,p (f ) p <, (2.23) D + where a,p (f ), D +, are the L p normalized wavelet coefficients defined in (2.12). n the case p = 2, the space B 2 is the same as L 2. We note that for p 1, Bp is a normed space, while for p<1instead of the triangle inequality we have that for every f, g B p f + g p B p f p B p + g p B p. (2.24) We will investigate the tree approximation of a given target function f in the metric of B p. We will use the abbreviated notation t n (f ) p := t n (f ) Bp. (2.25) We introduce a certain maximal function of the wavelet coefficients of a function f. We denote by T any finite subtree that has as its root and define ( 1 ) 1/p b := b,p (f ) := sup a,p (f ) p, (2.26) T #T T where the supremum is taken over all finite subtrees T with root. We modify ( b ) in order to obtain a decreasing tree sequence: b := b,p (f ) := inf b. (2.27) By the definition, we have b 1 b 2 if 1 2. We can use the new sequences (b (f )) D+ to define new function spaces. Given p, α, q, wedefineτ := (α + 1/p) 1. The space Xp,q α is by definition the set of all functions f B p such that f X α p,q := ( b,p (f ) ) D + lτ,q (2.28) is finite. n analogy to the approximation spaces A α q (L p) we define the approximation spaces A α q (B p, tree) as in (2.16) and (2.17) with σ n (f ) p replaced by t n (f ) p. Our main result is the following analogue of theorem 2.1. Theorem 2.3. For any α, q, p > 0, with equivalent norms. A α q (B p, tree) = X α p,q (2.29)

8 364 R.G. Baraniuk et al. / Near best tree approximation n the process of proving theorem 2.3 we will show that a certain thresholding procedure produces best tree approximants. Given ε > 0, define ε := ε (f ) := {: b (f)>ε} and N := N ε := # ε.wedefine T N := T N (f ) := A (f ). (2.30) ε (f ) Theorem 2.4. For any f B p,0<p<, andanyε > 0, the function T N (f ), N = N ε, is the best tree approximation to f with N nodes, that is, we have f TN (f ) Bp = t N (f ) p. (2.31) 3. Properties of the maximal sequences n this section, we will derive properties of the maximal sequence (b ). We start out by deriving related results for an arbitrary sequence (c ) D+ of nonnegative numbers such that c 0, 0. At the end of this section, we obtain the results we seek for the maximal sequence (b ) by taking c = a p,p. We fix a sequence (c ) D+ as described above. Given any set of dyadic cubes, we let Ave( ) denote the average of the c over. For each sequence (c ) D+,we define the maximal sequence c 1 := sup Ave(T ) = sup c, (3.1) T T #T T where the supremum is taken over all subtrees T with single root. We introduce some notation that we will utilize in the remainder of the paper. For any, we shall denote by T the largest finite subtree with root that satisfies c = Ave( T ). (3.2) The existence of such T follows from the fact that the c tend to zero as 0. The following lemma gives the uniqueness of T. Lemma 3.1. For each D +, there is a unique subtree T that satisfies (3.2) and has maximal cardinality. Proof. Suppose that T j, j = 1, 2, both satisfy (3.2) and that their common cardinality n is maximal. Let m be the cardinality of the tree T := T 1 T 2 having root. From definition (3.1), it follows that α := Ave(T ) c. From this it follows that β := Ave(T j \ T ) c, j = 1, 2. The tree T := T 1 T 2 has cardinality 2n m and root. t satisfies (2n m)ave(t ) = mα + 2(n m)β. Sincemα + (n m)β = nc, we have (2n m)ave ( T ) = 2nc mα (2n m)c. (3.3)

9 R.G. Baraniuk et al. / Near best tree approximation 365 n other words, Ave(T ) = c, and therefore 2n m n. T 1 = T 2. That is, n = m and Given any finite subtree T and any cube T,wedefine Ɣ := Ɣ (T ) := {K T : K }. (3.4) n other words, Ɣ is the largest subtree with root that is contained in T. We now derive properties of the subtrees T and the sequence (c ). Lemma 3.2. f D + and T,,andƔ = Ɣ (T ),then and Proof. Ɣ )>Ave(T ).SinceT T and c Ave(Ɣ ) Ave ( T ) = c, (3.5) Ave(Ɣ ) Ave ( T \ Ɣ ). (3.6) f (3.5) is not true, then Ave(T )>Ave(Ɣ ), which implies that Ave(T \ \ Ɣ is a tree with root, this contradicts the definition of. f (3.6) is not valid, then from (3.5) we find that Ave(T which again contradicts the definition of T The next lemma shows an ordering property for the subtrees T. Lemma 3.3. f D + and T,thenT Proof.. Ɣ (T ) and hence T T. \ Ɣ )>Ave(T ), Let Ɣ = Ɣ (T ) and let T := T Ɣ, which is a tree with root. Then, c Ave(Ɣ ) Ave ( T ) ( Ave T \ T ), (3.7) where the first inequality is by (3.5), and the second inequality is because Ave(T ) = c. For the last inequality, note that T \T can be written as a disjoint union of trees Ɣ K(T ), and hence we can apply (3.5) for each of these K. Now, (3.7) guarantees that if we form the new tree T := T (T \ T ), then the average over this tree will be at least as large as c.since(t \ T ) T =, the maximality of T gives that T \ T =. Hence, T Ɣ. Lemma 3.4. f D + and T,thenc c. Proof. We have c = Ave( T ) Ave(Ɣ ) c, (3.8) where the last inequality is (3.5).

10 366 R.G. Baraniuk et al. / Near best tree approximation From lemma 3.3, we see that any two of the T are either disjoint or one is con- T [1,2].1 is a supernode. Then {T } D is a tained in another. We call T Let D denote the set of all D + such that T partition of D +. a supernode if there is no such that T The set of supernodes has a natural ordering. We say T is a child of the supernode T say that the supernode T K T. Lemma 3.5. f the supernode T Proof. f c c,thent average of the (c K ) is c T if. Wealso if is the child of some cube is a child of the supernode T,thenc <c. T. This contradicts the maximality of T would be a tree with a single root on which the We shall apply the above results to the sequence b defined in (2.27). f we fix an f B p and take c := a,p (f ) p, then in view of the definitions (2.26) and (2.27), we have b p = c and b p = inf c. (3.9) t follows from the above that we have the following properties for the sequences ( b ) and (b ) [1,2]. Property 1. For each D +, b b whenever T. ndeed, this follows from lemma 3.4. Property 2. The sequence ( b ) D is a decreasing sequence. That is, if, D,and,,then b < b. This follows from lemma 3.5. Moreover, from these two properties, we conclude that: Property 3. For each D and T,wehave ( 1 1/p b = b = b = #T ak) p. (3.10) That is, (b ) is constant on each supernode. K T 1 The notion of solving the optimal tree approximation problem by grouping nodes into supernodes and sorting arose first in the context of optimizing the time-frequency representation of a signal. The solution paralleled here plus a fast computational algorithm are given in [2]. n [1], the algorithm was applied to the wavelet representation and a particular wavelet denoising problem. However, no approximation properties were derived. See also [12] for an interesting connection with the classification and regression trees (CART) framework..

11 R.G. Baraniuk et al. / Near best tree approximation 367 Also from the definition (3.9) of the b,wehave: Property 4. f,thenb b. t also follows from property 3 that a,p (f ) p = #T b p = b p. (3.11) T T Since the supernodes T, D form a partition of D,wehave a,p (f ) p = b p. (3.12) D D Finally, we have: Property 5. f T, D, is any supernode, then for any tree T T with root, we have T a p,p #T b p. (3.13) ndeed, the left side of (3.13) does not exceed #T b p definition of b and property 3. = #T b p, because of the 4. Best tree approximation There are certain values of N for which we can find the best tree approximation to a function f B p with N nodes. To describe these situations, we fix 0 < p<, letε>0beanygiven positive number, and consider the set ε := ε (f ) = {: b (f)>ε}. t follows from properties 2 and 3 of the previous section that ε is a union of supernodes: ε = Dε T with Dε D. From property 4, it follows that the set ε forms a tree. From the fact that f B p, it follows that ε has finite cardinality N ε = # ε. For such values of N = N ε,wedefine T N (f ) := ε (f ) A (f ), (4.1) which is a tree approximation to f with N nodes. We now prove theorem 2.4. Proof of theorem 2.4. Let T be any tree with cardinality N and let T := T A (f ). We shall show that T does not approximate f as well as T N.

12 368 R.G. Baraniuk et al. / Near best tree approximation n the case where ε T =, that is, when T and ε do not have common roots, then T can be written as the disjoint union of subtrees T, with roots D \ Dε,and such that T T. t follows from property 5 of the last section that a K,p (f ) p #T b p = #T b p <εp #T, (4.2) K T where the last inequality follows from the fact that is not in Dε. On the other hand, since ε is a union of supernodes, for every Dε a K,p (f ) p = #T b p = #T bp >εp #T. (4.3) K T From (4.2) and (4.3) and taking into account the fact that T and ε have the same cardinality, we have that a p K,p < a p K,p. (4.4) K ε This gives K T f T N p B p = f p B p a p K,p < f p B p a p K,p = f T p B p, (4.5) K ε K T as desired. For the rest of the proof for any tree T we shall denote by R(T ) the set of its roots. Let us now assume that T and ε have common roots, in other words R(T ε ) = R(T ) R( ε ).ft 0 := T ε, then we can write the set T := T \ T 0 as the union T = T 1 T 2,whereT 1 has all of its roots in R(T ) \ R(T 0 ) and (using remark 2.2) T 2 = T, where each T,, is a child of ε and hence it is the root of a supernode. As in (4.2), we have K T 1 a K,p (f ) p ε p #T 1. (4.6) Similarly, using property 5 of the last section, we obtain a K,p (f ) p #T b p = #T bp <εp #T 2, (4.7) = K T 2 K T where the last inequality follows from the fact that each is not in Dε. On the other hand, consider the set T := ε \ T 0. This set can be written as a union of subtrees T,,whereT = Ɣ T for some supernode T, D ε,and some T. t follows from lemma 3.2 that K T a K,p (f ) p ε p #T. (4.8)

13 R.G. Baraniuk et al. / Near best tree approximation 369 Now, T \ T 0 and ε \ T 0 have the same cardinality and therefore, if these sets are not empty, we will have a p K,p < a p K,p. (4.9) K ε Again, this gives K T which completes the proof of the theorem. f T N p B p < f T p B p, (4.10) 5. Tree approximation spaces n this section, we shall prove theorem 2.3 in two steps. We continue to use the notation of the last section for the sets ε. We denote by b n := b n (f ) the nth largest of the numbers b (f ), D +. We can identify each each b n with a node D + such that b = b n. t may happen that many b take the same value b n. n this case we order the b as follows. The first priority is given to the supernode (in its natural order), and then the priority within the supernode goes according to the tree structure within that supernode. n other words, we have b n1 = b 1,..., b }{{} n2,b n2 +1,..., b }{{} n3,..., (5.1) 1st family of supernodes 2nd family of supernodes where b nk b nk+1 and b i = b j if n k i, j < n k+1, k = 1, 2,... With this correspondence, to each positive integer N, we have associated a tree T N with N nodes, and the sets {b } TN and {b n } N n=1 are identical. The tree T N+1 is obtained from T N by adding one node (growing T N ). We define T N := T N (f ) := A (f ). (5.2) T N First, we begin by deriving the inclusion Xp,q α Aα q (B p, tree). Theorem 5.1. Let 0 <p<, 0<q, andα > 0. f f Xp,q α,thenf A α q (B p, tree) and ( ) [ ] 1/q f A α q (B p,tree) C 2 nα q t 2 n(f ) p C f X α, (5.3) p,q n 1 where t 1/2 (f ) := f p,andc depends only on α and p. Proof. We consider only the case q <. The case q = requires some trivial modifications. The left inequality follows directly from the definition of the norm on A α q (B p, tree) and the monotonicity of the sequence (t n ). n order to establish the right

14 370 R.G. Baraniuk et al. / Near best tree approximation inequality in (5.3), it suffices to prove that for each n 1, the following weak-type inequality holds ( ) ( ) 1/p t 2 n(f ) p C 2 j/p p b 2 j, (5.4) j n where for the purposes of this formula we define b 1/2 := 0andt 1/2 = t 0. ndeed, if (5.4) holds, then from Hardy s inequality [11, p. 27], we find ( ) 1/q ( 2 nα ) ) 1/q q ( t 2 n(f ) p C( 2 nα 2 n/p ) q b 2 n n= 1 n=0 ( ) ( ) 1/q = C 2 n/τ q b 2 n C f X α, (5.5) p,q n=0 where τ := (α + 1/p) 1. We now prove (5.4). For n = 1, this inequality follows from the definition of p and the monotonicity of the sequence (b n ). For the case, n 0, we let N = 2 n and assume that n k N<n k+1.then f T N p B p a,p (f ) p + a,p (f ) p =: S 1 + S 2. (5.6) T nk+1 1\T N / T nk+1 1 From (3.11), we have S 2 = b p j, (5.7) j=n k+1 and therefore, using the monotonicity of the coefficients, we easily obtain S 2 b p j 2 j b p. (5.8) 2 j j=2 n j=n To estimate S 1, note that S 1 T nk+1 1\T nk 1 a,p (f ) p = n k+1 1 j=n k b p j = (n k+1 1)b p n k+1 1, (5.9) where the first equality uses (3.12) and the last inequality uses that b j is constant in the range of the right sum. The right side does not exceed 2 2 m b p 2m,wherem n is the largest integer such that 2 m n k+1 1. Hence, S 1 C 2 j b p. (5.10) 2 j j=n

15 R.G. Baraniuk et al. / Near best tree approximation 371 Putting together (5.6), (5.8) and (5.10), we obtain (5.4), which concludes the proof of the theorem. Next we shall prove the following converse to theorem 5.1. Theorem 5.2. Let 0 <p<, 0<q, andα>0. f f A α q (B p, tree) then f X α p,q and f C f Xα p,q A α q (B p,tree), (5.11) with C depending only on α and p. Proof. Again we shall treat only the case 0 <q<. Letf B p,andlet(b n ) n 1 be the rearranged sequence of (b ) D+ as introduced earlier. We define m k := [(n k 1)/2], k = 1, 2,... t follows that m 1 = 0 and that no more than two consecutive values of m k are equal. We first claim that for every k = 2, 3,...we have b nk 1 Cn 1/p k t mk (f ) p. (5.12) To prove (5.12), we fix a value of k, taken := m k and consider the best tree T of cardinality N. LetT = T A (f ) be the corresponding approximant. Consider also the tree bnk = with # bnk = n k 1. (5.13) D bn k T Then, the set bnk \ T has at least N elements, and we can write bnk \ T = [ T \ ( T T )]. (5.14) D bn k Going further, for each D b nk, we can use remark 2.2 to write T \ ( ) T T = T, (5.15) where T = Ɣ (T ) (see (3.4)). Thus, T is a subtree with root T. t follows from lemma 3.2 and property 3 of section 3 that for each,wehave a p K,p bp #T bn p k 1 #T. (5.16) K T f we sum these inequalities over all andthenoverall D b nk, we obtain t N (f ) p = f T p B p a p K,p bp n k 1 #T bn p k 1 N. K T Dbn k Dbn k (5.17)

16 372 R.G. Baraniuk et al. / Near best tree approximation This concludes the proof of (5.12). Using (5.12) it follows immediately that f q Xp,q α = 1 [ ν 1/τ ] q b ν = ν k=2 n k 1 ν<n k k=2 C q C q k=2 k=2 n q/p k t mk (f ) q p t mk (f ) q p n k 1 ν<n k ν (q/τ) 1 n k 1 ν<n k ν (q/τ q/p) 1 bn q k 1 ν (q/τ) 1 n k 1 ν<n k = C q t mk (f ) q p ν αq 1, (5.18) k=1 n k 1 ν<n k where here and later in this proof C represents a constant which depends at most on α and p. Now, ν αq 1 C q ν αq 1 (5.19) n k 1 ν<n k m k 2 ν<m k for each k = 2, 3,... Using this and the fact that t ν (f ) p, ν = 0, 1,..., is decreasing, we obtain t mk (f ) q p ν αq 1 C q ν αq 1 t ν (f ) q p k=1 n k 1 ν<n k k=1 m k 2 ν<m k C q n=0 Combining (5.18) and (5.20), we obtain the desired result. [ (n + 1) α t n (f ) p ] q 1 n + 1. (5.20) Proof of theorem 2.3. Employing theorem 5.1, we immediately obtain the inclusion, while the reverse relationship follows from theorem 5.2. References [1] R. Baraniuk, Optimal tree approximation using wavelets, in: Proc. SPE Tech. Conf. on Wavelet Applications in Signal Processing V, Vol. 3813, Denver, uly [2] R. Baraniuk and D. ones, A signal-dependent time-frequency representation: Fast algorithm for optimal kernel design, EEE Trans. Signal Processing 45(1) (1994) [3]. Bergh and. Löfström, nterpolation Spaces (Springer, New York, 1976). [4] A. Cohen, W. Dahmen,. Daubechies and R. DeVore, Tree approximation and optimal encoding, M Preprint Series, University of South Carolina, No. 9 (1999). [5] A. Cohen,. Daubechies and.-c. Feauveau, Biorthogonal bases of compactly supported wavelets, Comm. Pure Appl. Math. 45 (1992)

17 R.G. Baraniuk et al. / Near best tree approximation 373 [6] A. Cohen,. Daubechies, O. Guleryuz and M. Orchard, On the importance of combining wavelet based non-linear approximation in coding strategies, preprint, Princeton University (1998). [7] A. Cohen, R. DeVore and R. Hochmuth, Restricted approximation, Constructive Approximation 16 (2000) [8]. Daubechies, Orthonormal bases of compactly supported wavelets, Comm. Pure Appl. Math. 41 (1988) [9]. Daubechies, Ten Lectures on Wavelets (SAM, Philadelphia, 1992). [10] R. DeVore, B. awerth and V. Popov, Compression of wavelet decompositions, Amer.. Math. 114 (1992) [11] R. DeVore and G.G. Lorentz, Constructive Approximation (Springer, New York, 1993). [12] D.L. Donoho, CART and best-ortho-basis: A connection, Ann. Statist. 25(5) (1997)