Course Notes for Greedy Approximations Math Th. Schlumprecht

Transcription

1 Course Notes for Greedy Approxmatons Math Th. Schlumprecht March 10, 2015

2 2

3 Contents 1 The Threshold Algorthm Greedy and Quas Greedy Bases The Haar bass s greedy n L p [0, 1] and L p R) Quas greedy but not uncondtonal Greedy Algorthms In Hlbert Space Introducton Convergence Convergence Rates Greedy Algorthms n general Banach Spaces Introducton Convergence of the Weak Dual Chebyshev Greedy Algorthm Weak Dual Greedy Algorthm wth Relaxaton Convergence Theorem for the Weak Dual Algorthm Open Problems Greedy Bases Greedy Algorthms Appendx A: Bases n Banach spaces Schauder bases Markushevch bases Appendx B: Some facts about L p [0, 1] and L p R) The Haar bass and Wavelets Khntchne s nequalty and Applcatons

4 4 CONTENTS Sgnals or mages are often modeled as elements of some Banach space consstng of functons, for example CD), L p D), or more generally Sobolev spaces W r,p D), for a doman D R d. These functons need to be processed : approxmated, converted nto an obect whch s storable, lke a sequence of numbers, and then reconstructed. Ths means to fnd an approprate bass of the Banach space, or more generally a dctonary and to compute as many coordnates of the gven functons wth respect to ths bass as necessary to satsfy the gven error estmates. Now the queston one needs to solve, s to fnd the coordnates one wants to use, gven a restrcton on the budget. Defnton Let X always) be a separable and real Banach space. We call D S X a dctonary of X f spand) s dense and x D mples that x D. An approxmaton algorthm s a map G : X spand) N, x Gx) = G n x)), wth the property that for n N and x X, there s a set Λ n,x) D of cardnalty at most n so that G n x) spanλ n ). For n N we call G n x) the n-term approxmaton of x. Usually G n x) s computed nductvely by maxmzng a certan value, therefore these algorthms are often called greedy algorthms. Remark. If X has a bass e n ) wth borthogonals e n) and G = P n ), where P n s the n-th canoncal proecton, would be an example of an approxmaton algorthm. Nevertheless the pont s to be able to adapt the set Λ n to the vector x, and not lettng t be ndependent. of x. The man questons are 1) Does G n x)) converge to x? 2) If so, how fast does t converge? How fast does t converge for certan x? 3) How does x G n x) compare to the best n-term approxmaton defned by σ n x) = σ n x, D) = nf nf z x? Λ D,#Λ=n z spanλ)

5 Chapter 1 The Threshold Algorthm 1.1 Greedy and Quas Greedy Bases We start wth the Threshold Algorthm: Defnton Let X be a separable Banach space wth a normalzed M- bass e, e ) : N) ; we mean by that e = 1, for N) For n N and x X let Λ n N so that mn e x) max e x), Λ n N\Λ n.e. we are reorderng e x)) nto e σ) x)), so that and put for n N Then defne for n N e σ 1 x) e σ 2 x) e σ 3 x)..., Λ n = {σ 1, σ 2,... σ n }. G T n x) = G T n ) s called the Threshold Algorthm. Λ n e x)e. Defnton A normalzed M-bass e ) s called Quas-Greedy, f for all x QG) x = lm n GT n x). A bass s called greedy f there s a constant C so that G) x G T x) Cσ n x), where we defne σ n x) = σ n x, e ) ) = nf nf Λ N,#Λ=n z spane : Λ) 5 z x.

6 6 CHAPTER 1. THE THRESHOLD ALGORITHM In that case we say that e ) s C-greedy. We call the smallest constant C for whch G) holds the greedy constant of e n ) and denote t by C g. Remarks. Let e, e ) : N) be a normalzed M bass. 1. From the property that e n ) s fundamental we obtan that for every x X σ n x) n 0, t follows therefore that every greedy bass s quas greedy. 2. If e ) s an uncondtonal bass of X, and x = =1 a X, then x = lm n n a π) e π), for any permutaton π : N N and thus, n partcular, also for a greedy permutaton,.e. a permutaton, so that a π1) a π2) a π3).... Thus, an uncondtonal bass s always quas-greedy. 3. Schauder bases have a specal order and mght be reordered so that the cease to be bass. But uncondtonal bases, M bases, quas greedy M-bases, greedy bases keep ther propertes under any permutaton, and can therefore be ndexed by any countable set. 4. In order to obtan a quas greedy M-Bass whch s not a Schauder bass, one could take quas greedy Schauder bass, whch s not uncondtonal ts exstence wll be shown later), but admts a sutable reorderng under whch s not a Schauder bass anymore. Nevertheless, by the observatons n 3), t wll stll be a quas greedy M-bass. But t seems unknown whether or not there s a quas greedy M-bass whch cannot be reordered nto a Schauder bass. Examples greedy. 1. If 1 p <, then the unt vector bass e ) of l p s

7 1.1. GREEDY AND QUASI GREEDY BASES 7 2. The unt vector bass e ) n c 0 s 1-greedy. 3. The summng bass s n of c 0 s n = n e ) s not quas greedy. 4. The unt bas of l p l q ) 1 s not greedy but 1-uncondtonal and thus quas greedy). Proof. To prove 1) let x = x e l p, and let Λ n N be of cardnalty n so that mn{ x : Λ n } max{ x : N \ Λ n } and Λ N be any subset of cardnalty n and z = z e l p wth suppz) = { N : z 0} Λ. Then x z p p = Λ x z p + x p N\Λ x z p + x p Λ N\Λ n x p = G T x) x p. N\Λ n Thus σ n x) = nf{ z x p : #suppz) n} = G T x) x p. 2) can be shown n the same way as 1). In order to show 3) we choose sequences ε ) 0, 1), n ) N as follows: ε 2 = 2 and ε 2 1 = ), 3 for N and Note that the seres n = 2 and N = n n for N 0. =1 x = = N =N 1 +1 N =N 1 +1 ε 2 1 s 2 1 ε 2 s 2 ) ε2 1 ε 2 )s 2 1 ε 2 e 2 )

8 8 CHAPTER 1. THE THRESHOLD ALGORITHM converges, because and N =N 1 +1 ε 2 e 2 c 0 N =N 1 +1 ε 2 1 ε 2 )s 2 1 = n ε 2 1 ε 2 ) = Now we compute for l N 0 the vector x G T 2N l +n l+1 x): x G T 2N l +n l+1 x) = N l+1 =N l +1 ε 2l+2 s 2 + From the monotoncty of s ) we deduce that N =l+2 =N 1 +1 N =l+2 =N 1 +1 ε 2 1 s 2 1 ε 2 s 2 x. 1 2 <. ε 2 1 s 2 1 ε 2 s 2 ). However, N l+1 =N l +1 N l+1 ε 2l+2 s 2 = =N l +1 ε 2l+2 = l + 1 l, whch mples that G T n x) s not convergent. To show 4) assume w.l.o.g. p < q, and denote the unt vector bass of l p by e ) and the unt vector bass of l q by f ) for n N and we put Thus Nevertheless G T n xn)) = xn) = n 1 2 e + n f. n f, and thus G T n xn)) xn) = 1 2 n1/p. x n 1 2 e = n 1/q, and snce 1 2 n1/p /n 1/q, for n, the bass {e : N} {f : N} cannot be greedy.

9 1.1. GREEDY AND QUASI GREEDY BASES 9 Remarks. Wth the arguments used n 4) Examples one can show that the usual bases of ) n=1 ln q l p and l p l q ) = n=1 l ) q l p are also not greedy but of course uncondtonal. Now n [BCLT] t was shown that l p l q has up to permutaton and up to somorphc equvalence a unque uncondtonal bass, namely the one ndcated above. Snce, as t wll be shown later, every greedy bass must be uncondtonal, the space does not have any greedy bass. Due to a result n [DFOS] however ) n=1 ln q l p has a greedy bases f 1 < p, q <. More precsely, the followng was shown: Let 1 p, q. a) If 1 < q < then the Banach space n=1 ln p ) lq has a greedy bass. b) If q = 1 or q =, and p q, then n=1 ln p ) lq has not a greedy bass. Here we take c 0 -sum f q =. The queston whether or not l p l q ) has a greedy bass s open and qute an nterestng queston. The followng result by Wotaszczyk can be seen the analogue of the characterzaton of Schauder bases by the unform boundedness of the canoncal proectons for quas-greedy bases. Theorem [Wo2] A bounded M-bass e, e ), wth e = 1, N, of a Banach space X s quas greedy f and only f there s a constant C so that for any x X and any m N t follows that 1.1) G T mx) C x We call the smallest constant so that 1.1) s satsfed the Greedy Proecton Constant. Remark. Theorem s bascally a unform boundedness result. Nevertheless, snce the G T m are nonlnear proectons we need a drect proof. We need frst the followng Lemma: Lemma Assume there s no postve number C so that G T mx) C x for all x X and all m N. Then the followng holds: For all fnte A N all K > 0 there s a fnte B N, whch s dsont from A and a vector x, wth x = B x e, such that x = 1 and G T mx) K, for some m N. Proof. For a fnte set F N, defne P F to be the coordnate proecton onto spane : F ), generated by the e ),.e. P F : X spane : F ), x P F x) = F e x)e.

10 10 CHAPTER 1. THE THRESHOLD ALGORITHM Snce there are only fntely many subsets of A we can put M = max P F = max sup e x)e e e <. F A F A x B X F A Let K 1 > 1 so that K 1 M)/M +1) > K, and choose x 1 S X spane : N) and k N so that so that G T k x 1) K 1. We assume wthout loss of generalty after sutable small perturbaton) that all the non zero numbers e nx 1 ) are dfferent from each other. Then let x 2 = x 1 P A x 1 ), and note that x 2 M + 1 and G T k x 1) = G T mx 2 ) + P F x 1 ) for some m k and F A. Thus G T mx 2 ) K 1 M, and f we defne x 3 = x 2 / x 2, we have G T k x 3) K 1 M)/M + 1) > K. It follows that the support B of x = x 3 s dsont from A and that G T mx) > K. Proof of Theorem Let b = sup e. Assume there s no postve number C so that G T mx) C x for all x X and all m N. Applyng Lemma we can choose recursvely vectors y 1, y 2,... n S X spane : N) and numbers m n N, so that the supports of the y n, whch we denote by B n, are parwse dsont, Recall that for z = =1 z e, we call suppz) = { N : e z) 0}, the support of z) and so that n 1 1.2) G T m n y n ) 2 n b n ε 1, where Then we let ε = mn { 2, mn{ e y ) : B } }. x = n 1 n=1 ) ε /b) y n, whch clearly converges) and wrte x as Snce e y ) b, for, N { mn x : n } B n 1 x = x e. ε b ε n = n ε b b max { x : N \ n B }.

11 1.1. GREEDY AND QUASI GREEDY BASES 11 We may assume w.l.o.g. that m n #B n, for n N. Lettng k = m + 1 =1 #B, t follows that and thus by 1.2) G T k x) G T k x) = GT m 1 1 =1 1 ) ε s /b) )y + G T m ε /b) y ). s=1 ) 1 ) 1 ε /b) y =1 =1 =1 1 ) ε s /b) y 2 b, whch mples that G T k does not converge. Let C > 0 such that G T mx) C x for all m N and all x X. Let x X and assume w.l.o.g. that suppx) s nfnte. For ε > 0 choose x 0 wth fnte support A so that x x 0 < ε. Usng small perturbatons we can assume that A suppx) and that A suppx x 0 ). We can therefore choose m N large enough so that G T mx) and G T mx x 0 ) are of the form s=1 G T mx) = B e x)e and G T mx x 0 ) = B e x x 0 )e wth B N such that A B. It follows therefore that x G T mx) x x 0 + x 0 G T mx) = x x 0 + G T mx 0 x) 1 + C)ε, whch mples our clam by choosng ε > 0 to be arbtrarly small. Defnton An M bass e, e ) s called uncondtonal for constant coeffcents f there s a postve constant C so that for all fnte sets A N and all σ n : n A) {±1} we have 1. e n σ n e n C e n C n A n A Proposton A quas-greedy M bass e n, e n) s uncondtonal for constant coeffcents. Actually the constant n Defnton can be chosen to be equal to twce the proecton constant n Theorem Remark. We wll show later that there are quas-greedy bases whch are not uncondtonal. Actually there are Banach spaces whch do not contan any uncondtonal basc sequence, but n whch every normalzed weakly null sequence contans a quas-greedy subsequence. n A

12 12 CHAPTER 1. THE THRESHOLD ALGORITHM Proof of Proposton Let A N be fnte and σ n : n A) {±1}. Then f we let δ 0, 1) and put m = #{ A : σ = +1} we obtan n A,σ n=+1 e n = G T m C n A,σ n=+1 n A,σ n=+1 e n + e n + n A,σ n= 1 n A,σ n= 1 By takng δ > 0 to be arbtrarly small, we obtan that. e n C e n n A,σ n=+1 n A 1 δ)e n ) 1 δ)e n. Smlarly we have and thus, n A,σ n= 1, e n C e n n A. σ n e n 2C e n n A n A We now present a characterzaton of greedy bases obtaned by Konyagn and Temlakov. We need the followng notaton. Defnton We call a a normalzed basc sequence democratc f there s a constant C so that for all fnte E, F N, wth #E = #F t follows that 1.3) e C e E F In that case we call the smallest constant, so that 1.3) holds, the Constant of Democracy of e ) and denote t by C d. Theorem [KT1] A normalzed bass e n ) s greedy f and only t s uncondtonal and democratc. In ths case 1.4) maxc s, C d ) C g C d C s C 2 u + C u, where C u s the uncondtonal constant and C s s the suppresson constant. Remark. The proof wll show that the frst nequalty s sharp. Recently t was shown n [DOSZ1] that the second nequalty s also sharp.

13 1.1. GREEDY AND QUASI GREEDY BASES 13 Proof of Theorem Let x = e x)e X, n N and let η > 0. Choose x = Λ n a e so that #Λ n = n whch s up to η the best n term approxmaton to x snce we allow a to be 0, we can assume that #Λ s exactly n),.e. 1.5) x x σ n x) + η. Let Λ n be a set of n coordnates for whch b := mn Λn e x) max N\Λ n e x) and G T n x) = Λ n e x)e. We need to show that x G T n x) C d C s C 2 u + C u )σ n x) + η). Then x G T n x) = e x)e = N\Λ n e x)e + e x)e. Λ n \Λn N\Λ n Λn) But we also have e 1.6) x)e bc u Λ n \Λn bc u C d Λ n \Λn e Λ \ nλ n By Proposton ) e [Note that #Λ n \ Λ n) = #Λ n \ Λ n )] CuC 2 d e x)e Λ n\λ n [Note that e x) b f Λ n \ Λ n ] C s CuC 2 d e x) a )e + e x)e Λ n N\Λ n = C s C 2 uc d x x C s C 2 uc d σ n x) + η) and 1.7) N\Λ n Λ n) e x)e Cs e x) a )e + Λ n N\Λ n = C s x x C s σ n x) + η). e x)e Ths shows that e ) s greedy and, snce η > 0 s arbtrary, we deduce that C g C s C 2 uc d + C s.

14 14 CHAPTER 1. THE THRESHOLD ALGORITHM Assume that e ) s greedy. In order to show that e ) s democratc let Λ 1, Λ 2 N wth #Λ 1 = #Λ 2. Let η > 0 and put m = #Λ 2 \ Λ 1 ) and Then t follows e = x G T m x) Λ 1 x = Λ 1 e η) Λ 2 \Λ 1 e. C g σ m x) snce e ) s C g -greedy) x C g C g e η) Λ 1 Λ 2 Λ 1 \Λ 2 e Snce η > 0 can be taken arbtrary, we deduce that e Cg e. Λ 1 Λ 2 Λ 2 \Λ 1 e Thus, t follows that e ) s democratc and C d C g. In order to show that e ) s uncondtonal let x = e x)e X have fnte support S. Let Λ S and put y = Λ e x)e + b S\Λ wth b > max S e x). For n = #S \ Λ) t follows that G T n y) = b e, S\Λ and snce e ) s greedy we deduce that note that #suppy x) = n) e x)e = y G T n y) C g σ n y) C g y y x) = C g x, Λ e,. whch mples that e ) s uncondtonal wth C s C g.

15 1.2. THE HAAR BASIS IS GREEDY IN L P [0, 1] AND L P R) The Haar bass s greedy n L p [0, 1] and L p R) Theorem For 1 < p < there are two constants c p C p, dependng only on p, so that for all n N and all A T wth #A = n c p n 1/p h p) t C p n 1/p. t A In partcular h p) t ) t T s democratc n L p [0, 1]. Wth Theorem and Theorem we deduce that Corollary The Haar Bass of L p [0, 1], 1 < p < s greedy. The proof wll follow from the followng three Lemmas. Lemma For any 0 < q < there s a d q > 0 so that the followng holds. Let n 1 < n 2 <... n k be ntegers and let E [0, 1] be measurable for = 1,... k. Then we have Proof. Defne 1 0 k ) qdx k 2 n/q 1 E x) dq 2 n me ). fx) = k 2 n/q 1 E x). For = 1,... k wrte E = E \ k =+1 E. It follows that for x E fx) n 2 n/q 2 /q = 2n+1)/q 1 2 1/q 1 =1 =1 21/q 2 1/q 1 } {{ } d 1/q q 2 n /q. Thus 1 whch fnshes the proof. 0 fx) q dx d q k k 2 n me ) d q 2 n me ), =1 Lemma For 1 < p < there s a C p > 0 so that for all n N, A T wth #A = n, and ε t ) { 1, 1} t follows that ε t h p) t C p n 1/p. p t A

16 16 CHAPTER 1. THE THRESHOLD ALGORITHM Proof. Abbrevate h t = h p) t for t T. Let n 1 < n 2 <... < n k be all the ntegers n for whch there s a t A so that msupph t )) = 2 n. For = 1,... k put E = supph,n )). Snce and thus {0,1,...2 n 1},n,) A me ) = 2 n #{ {0, 1,... 2 n 1}, n, ) A} #{ {0, 1,... 2 n 1}, n, ) A} = 2 n me ). It follows therefore that { k n = #{ {0, 1,... 2n 1}, n, ) A} = k 2n me ) f 0 A 1 + k 2n me ) f 0 A. Assume wthout loss of generalty that 0 A. It follows that [ ] 1 [ p k ] 1/p [ pdx k ] 1/p ε t h t = 2 n/p 1 E d 1/p p 2 n me ) = d 1/p p n 1/p. t A 0 [d p as n Lemma 1.2.3] Lemma For 1 < p < there s a c p > 0 so that for all n N, A T wth #A = n, and ε t ) { 1, 1} t follows that ε t h p) t c p n 1/p. p t A Proof. Note that for 1 < p, q < wth 1 p + 1 q and s, t T t follows that h p) t, h q) = δt, s), s thus the clam follows from the fact that the h p) t s are normalzed n L p [0, 1] and by Lemma usng the dualty between L p [0, 1] and L q [0, 1]. Indeed, t A ε t h p) t t A ε t h p) t, n = t A ε th q) t t A ε th q) t t A ε th q) n1/p c q, where c q s chosen lke n Lemma Our clam follows therefore bu lettng C p = 1/c q. t

17 1.3. QUASI GREEDY BUT NOT UNCONDITIONAL A quas greedy bass of L p [0, 1] whch s not uncondtonal In ths secton we make the general assumpton on a separable Banach space X, that X has a normalzed bass e n ) whch s Besselan meanng that for some constant C B 1.8) x = e 1 x)e e C x) 2) 1/2 for all x X. B where e ) denote the coordnate functonals for e ) We secondly assume that e ) has a subsequence e m : N) whch s Hlbertan whch means that for some constant C H 1.9) e m x)e m CH e m x) 2) 1/2 for all x spanem : N). Example An example for such a basc sequence are the trgonometrcally polynomal t n : n Z) n L p [0, 1] wth p > 2. Indeed, for a n : n N) C t follows from Hölder s or Jensen s) nequalty that ) 1 N 1/p ) a e nξ/2π p 1 N 1/2 dξ a e nξ/2π 2 N dξ = a 2) 1/2. 0 n= N 0 n= N n= N Secondly t follows from the complex verson of Khntchne s nequalty Theorem 6.2.4) that the subsequence t 2 n : n N) of the trgonometrc polynomals s equvalent to the l 2 -unt vector bass. We recall the 2 n by 2 n matrces A n) = a n),) : 1, 2n ), for n N, whch were ntroduced n Secton 5.2. Let us recall the followng two propertes whch we wll need here: 1.10) 1.11) A n) s untary operator on l 2n 2, and a n),1) = 2 n/2. For k N we put n k = 2 2k and B k) = b k), : 1, n k) = A 2k) actng on l n k 2 ), for k N whch mples that n k+1 = n 2 k. We let h : N) = e m : N) and f : N) = e s : s N \ {m : N}), so that f f = e s and f = e t then then < f and only f s < t. For k N we defne a famly g k) : = 1, 2,... n k ) as follows g k) 1 = f k and g k) = h Sk 1 + 1, for = 2, 3,... n k,

18 18 CHAPTER 1. THE THRESHOLD ALGORITHM where S 0 = 0, and, nductvely, S = S 1 + n 1. If we order g k) : k N, = 1, 2,... n k ) lexcographcally we note that the sequence s equal to the sequence 1) g 1, g1) 2,... g1) n 1, g 2) 1,..., g2) n 2, g 3) 1,,... ) f1, h 1, h 2,... h n1 1, f 2, h n1,... h n2 2, f 3,... ). Then we defne for k N a new system of elements ψ k) 1.12) or, n other words, ψ k) = ψ k) 1 g k) 1 ψ k) 2. = g k) Bk) 2. ψ n k) k g n k) k n k Our goal s now to prove the followng result b k),) gk) for = 1, 2... n k. : = 1, 2... n k ), by Theorem Ordered lexcographcally, the system ψ k) : k N, = 1, 2... n k ) s a quas-greedy bass of X. Proposton Ordered lexcographcally, g k) Besselan bass of X. : k N, = 1, 2... n k ) s a Proof. Gven that g k) : k N, = 1, 2... n k ) s a reorderng of e ), whch was assumed to be a Besselan bass of X, we only need to show that g k) : k N, = 1, 2... n k ) s a basc sequence. To do so we need to show that there s a constant C 1 so that for all N N, all M {1, 2... n M } and all c k) : k N, = 1, 2... n k ), wth beng fnte, t follows #{k, ) : k N, = 1, 2... n k, c k) 0} 1.13) N 1 n k k=1 c k) g k) + M c N) g N) n k C k=1 c k) g k).

19 1.3. QUASI GREEDY BUT NOT UNCONDITIONAL 19 Snce the g k) are a reorderng of the orgnal bass e ) we can wrte x = n k c k) g k) as x = k=1 c e, =1 where c = c k) f e = g k) and for each N there s exactly one such choce of k and {1, 2... n k }). From 1.8) and 1.9) we deduce that 1.14) N 1 n k k=1 =2 c k) g k) + M =2 c N) g N) N 1 n k C H k=1 =2 c k) 2 + M =2 c N) 2) 1/2 C H c 2) 1/2 CH C B x. Snce g k) = f = e s, where s ) whch conssts of the elements of N \ {m : N}, ordered ncreasngly t follows that we can wrte N k=1 c k) 1 gk) 1 = s N c e {1,2,...s N }\{s : N} c e = k,) A c k) g k), for some set A {k, ) : k N, = 2, 3... n k }. If C e s the bass constant of e ) we deduce therefore that s N c e Ce x, and thus, usng 1.14), N c k) 1 gk) 1 k=1 Ths mples that N 1 n k k=1 s N c k) g k) + N c e + M c k) 1 gk) 1 k=1 c N) + k,) A g N) N 1 n k k=1 =2 c k) g k) c k) g k) C e + C B C H ) x + C B C H x whch mples our clam wth C = C e + 2C B C H. C e + C B C H ) x. + M =2 c N) g N) Proposton Under the lexcographcal order, ψ k) : = 1, 2... n k ) s a Besselan bass of X wth the same constant C B.

20 20 CHAPTER 1. THE THRESHOLD ALGORITHM Proof. We frst note that for k N X k = spanψ k) : = 1, 2... n k ) = spang k) : = 1, 2... n k ) and thus t follows that ψ k) : k N, = 1.2,... n k ) spans as g k) : k N, = 1.2,... n k ) a dense subspace of X. Secondly we observe that f wth and let or n g-coordnates: d k) We wrte x = k=1 x k wth Snce n k x k = d k) ψ k) = =1 ths means that or c k) d k) : k N, = 1, 2... n k ) K #{k, ) : k N, = 1, 2... n k, d k) 0} x = x = n k n k d k) Ψ k) k=1 n k c k) g k). k=1 x k = d k) ψ N) = n k =1 d k) n k b k),) gk) = n k n k c k) g k). g k) n k =1 b k),) dk) = : = 1, 2... n k ) = B k) ) 1 d k) : = 1, 2... n k ) : = 1, 2... n k ) = B k) )c k) : = 1, 2... n k ). n k c k) g k) If we proect x to ts frst, say L, coordnates n the lexcographcal order of Ψ k) : k N, = 1,... k n ), for N N and M n N, so that L = N 1 k=1 k n + M, ths proected vector equals to: N 1 n k k=1 d k) ψ k) + M d N) ψ N) = N 1 n k k=1 =2 c k) g k) + M d N) ψ N).

21 1.3. QUASI GREEDY BUT NOT UNCONDITIONAL 21 Therefore we only need to show that there s a constant C 1 so that for all k and all M n k 1.15) M d k) ψ k) n k C d k) ψ k) and that ψ k) ) s Besselan. It follows from the assumpton that the matrces B k) are untary and Proposton that x = k=1 x k 1 C B n k k=1 c k) 2) 1/2 = 1 C B n k k=1 d k) 2) 1/2, whch proves that ψ k) ) s Besselan. Secondly we note that 1.11) yelds M =1 d k) ψ k) = M =1 M d k) n k b k),) gk) d k) n 1/2 k g k) =1 M =1 M =1 n k =1 d k) d k) d k) 1 + M 2) 1/2 n k + =2 =1 g k) =2 d k) M =1 2) 1/2 n k M + CH 2) 1/2 n k + CH =1 =1 n k b k),) gk) =2 d k) b k),) d k) b k),) 2 ) 1/2 d k) 2) 1/2 By 1.10)) Therefore applyng 1.10) and then 1.8) t follows that M =1 d k) ψ k) whch proves our clam. 1 + C H ) n k =1 =1 d k) 2) 1/2 n k = 1 + C H ) c k) 2) 1/2 1 + CH )C B x k Our last step of provng Theorem s the followng

22 22 CHAPTER 1. THE THRESHOLD ALGORITHM Proposton ψ k) Proof. Let : = 1, 2... n k ) s quas-greedy. x = n k d k) ψ k) X, k=1 =1 wth x = 1 and suppose that the m-th greedy approxmate s gven by G T mx, Ψ) = d k) ψ k), k J I k where m = k J #I k. We need to show that there s a constant C 1 of course not dependent on x and m) so that 1.16) G T mx, Ψ) C x We wrte G T mx, Ψ) as G T mx, Ψ) = k J d k) I k ψ k) b k),1) f k) + k J } {{ } Σ 1 d k) I k b k),1) f k } {{ } Σ 2 recall that g k) 1 = f k ). From the defnton of the ψ k) we get that Σ 1 = k J d k) I k n k =2 ) b k),) gk) = k J n k =2 g k) ) d k) b k),), I k whch yelds by the choce of the g k), propertes 1.9), and 1.10) that. 1.17) Σ 1 C H k J = C H k J n k d k) I k =2 b k),) ) 1/2 2 [ B k)] 1 d k) : I k ) 2 2 ) 1/2 = C H d k) : I k ) 1/2 2) 2 By 1.10)) k J C H C B x By Proposton 1.3.4)). In order to estmate Σ 2 we splt I k, k N nto the followng subsets: I 1) k I 2) k = { I k : d k) n 1 k }} = { I k : d k) n 1/2 k }

23 1.3. QUASI GREEDY BUT NOT UNCONDITIONAL 23 and let I 3) k Σ s) 2 = k J From the defnton of I 1) k = { I k : n 1 k I s) k < d k) < n 1/2 } k d k) b k),1) f k for s = 1, 2, 3. and 1.11 t follows that d k) b k) n 1/2 I 1) k,1) k. and thus 1.18) Σ 1) 2 n 1/2 k 1. k J In order to estmate 2) Σ 2) 2 we we frst note that the defnton of I k yelds that and, thus, 1.19) Σ 2) 2 = k J k J k J #I 2) k )n 1 k I 2) k k J I 2) k n 1/2 k d k) b k) I 2) k,1) f k d k) 2 d k) By 1.11)) n 1/2 k #I 2) k )1/2 I 2) k d k) 2 I 2) k n k =1 d k) 2, C 2 B x 2 = C 2 B By Proposton 1.3.4)). d k) 2) 1/2 By Hölder s nequalty) Fnally we have to estmate Σ 3) 2. Before that let us make some observatons: We frst note that n the estmaton of Σ 1 we dd not use specfc propertes of the sets I k. Replacng n the estmaton of Σ 1 the sets I k by any set I k {1, 2... n k } n 1.17) and J by any set J N we obtan 1.20) k J I k d k) n k =2 ) b k),) gk) CH C B x

24 24 CHAPTER 1. THE THRESHOLD ALGORITHM Takng I k to be all of {1, 2... n k} and J = [1, K] for some K N we deduce from Proposton that 1.21) K n k k=1 =1 d k) b k),1) f k K n k k=1 =1 d k) ψ k) K n k + k=1 =1 d k) C Ψ x + C H C B x = C Ψ + C H C B, ψ k) b k),1) f k) where C Ψ denotes the bass constant of ψ k) ; k N, = 1, 2... n k ). Secondly we note that n the estmaton of Σ 1) k we could replace the set J by any subset J N and I 1) k by any subset to obtan 1.22) I k I1) k = { n k : d k) n 1 k }}, k J I k d k) b k),1) f k 1. Thrdly we note that n the estmaton of Σ 2) 2 n 1.19) we could have also replaced J by any subset of N, and for k N the set I 2) k by any subset to obtan 1.23) I k I2) k k J = { I k : d k) n 1/2 } k I k d k) b k) In order to estmate the Σ 3) 2 we defne,1) f k K = max{k J : I 3) k CB. 2 }, whch means that for some I 3) k t follows that d k) < n 1/2 K and note that for any k [1, K 1] ether k J or here we use the frst tme that we are dealng wth the threshold algorthm) k J, whch mples 1.24) d k) < n 1/2 K n 1 k for all {1, 2,... n k } here we are usng that n k+1 = n 2 k ) and thus for such a k the sets I3) k and I 2) k are empty.

25 1.3. QUASI GREEDY BUT NOT UNCONDITIONAL 25 We compute now Σ 3) 2 = I 3) K = I 3) K d K) b K),1) f K + d K) b K),1) f K + K 1 k=1 I 1) k k J,<K K 1 n k=1 =1 I 3) k d k) b k),1) f k d k) b k),1) f k d k) b k),1) f k k J,k<K I 2) k d k) b k),1) f k The frst term we estmate, usng Hölder s nequalty: I 3) K d K) b K),1) f K n 1/2 K n K d K) n 1/2 It follows therefore from 1.21), 1.22) and 1.23) K n1/2 K N K 3) Σ 2 CB + C Ψ + C H C B CB 2 d K) whch mples our clam lettng C = C B + C Ψ + C H C B C 2 B. 2) 1/2 CB. Corollary Apply Theorem to the trgonometrcal polynomals t n )= e 2πn ) : n Z) whch are a bass of L p [0, 1] and satsfy by Example the assumptons f p > 2. Ths leads to a quas greedy bass Ψ n : n N) of L p [0, 1]. Secondly note snce t n ) s absolutely bounded by 1n L [0, 1]), and snce the matrces B k), whch where used n the constructon of the bass Ψ n ) are unformly bounded as lnear operators on l n k, t follows that also Ψ n : n N) s bounded L [0, 1]. Ths mples by Corollary that Ψ n ) cannot be uncondtonal.

26 26 CHAPTER 1. THE THRESHOLD ALGORITHM

27 Chapter 2 Greedy Algorthms In Hlbert Space 2.1 Introducton We wll now replace n our greedy algorthms, bases by more general and possbly redundant systems. Let H always) be a separable and real Hlbert space. Recall that D S H s a dctonary of X f spand) s dense and x D mples that x D. An n-term approxmaton algorthm s a map G : H spand) N, x Gx) = G n x)), wth the property that for n N and x H, there s a set Λ n D of cardnalty at most n so that G n x) spanλ n ), G n x) s then called an n-term approxmaton of x. Perhaps the frst example was consdered by Schmdt [Schm]: Example [Schm] Let f L 2 [0, 1] 2 ),.e. f s a square ntegrable functon n two varables. By the Theorem of Arcela and Ascol we know that the set { n } D = u v : n N, u, v C[0, 1] s dense n C [0, 1] 2). Here we denote for two functons f, g : [0, 1] K f g : [0, 1] 2 K, x, y) fx)gy). Snce C [0, 1] 2) s dense n L 2 [0, 1] 2 ) t follows that { n } D = u v : n N, u, v L 2 [0, 1] 27

28 28 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE s dense n L 2 [0, 1] 2 ). The queston s now, how to fnd a good approxmate to f from D. E. Schmdt consdered the followng procedure and showed that t worked: Let f L 2 [0, 1] 2 ) and defne f 0 = f. Then choose u 1, v 1 L 2 [0, 1] so that f 0 u 1 v 1 2 = nf { f 0 u v 2 : u, v L 2 [01]}. Snce ths nfmum mght be hard to acheve he also consdered a weaker condton, and fxed some weakenng factor t 0, 1) and chose u 1, v 1 L 2 [0, 1] so that f 0 u 1 v t nf { f 0 u v 2 : u, v L 2 [01] }. Then he let f 1 = f 0 u 1 v 1. After n steps he obtaned u 1, v 1, u 2, v 2,... u n, v m L 2 [0, 1], and let f n = f n u v, and chose u n+1 and v n+1 n L 2 [0, 1] so that n+1 f n u n+1 v n+1 = f 0 u v 2 1 t nf { f n u v 2 : u, v L 2 [01] }. Fnally he proved that f n converges n L 2 [0, 1] 2 ) to 0 and thus G n f) = n+1 u v converges to f. He asked whether there s some general prncple behnd, and how and whether ths generalzes.. PGA) The Pure Greedy Algorthm. For x H we defne G n = G n x), for each n N 0, by nducton. G 0 = 0 and assumng that G 0, G 1... G n 1, have been defned for some n N we proceed as follows: 1) Choose z n D and a n R so that x G n 1 a n z n = 2) Put G n = G n 1 + a n z n. Note that for any x H t follows that nf x G n 1 az. z D,a R 2.1) nf x az 2 z D,a R

29 2.1. INTRODUCTION 29 [ = nf x 2 2a x, z + a 2 z 2 ] z D,a R [ = nf x 2 x, z 2 ] z D [a x 2 2a x, z +a 2 z 2 s mnmal for a= x, z ] = x 2 sup x, z 2. z D So condton 1) n PGA) can be replaced by the followng condton 1 ) 1 ) Choose z n D so that x G n 1, z n = sup x G n 1, z z D and 2) by 2 ) Put G n = G n 1 + x G n 1, z n z n. As already noted n Example 2.1.1, the sup n 1 ) PGA), respectvely the nf n 1) mght not be attaned or mght be hard to attan. In ths case we mght consder the followng modfcaton. WPGA) The Weak Pure Greedy Algorthm. We are gven a sequence τ = t n ) 0, 1). For x X we defne G n = G n x), for each n N 0, by nducton. G 0 = 0 and assumng that G 0, G 1... G n 1, have been defned for some n N we proceed as follows: 1) Choose z n D, so that x G n 1, z n t n sup x G n 1, z z D 2) Put G n = G n 1 + x G n 1, z n z n. For WPGA we call the sequence t n ) the weakness factors. A possbly faster but computatonal more laborous) algorthm s the followng Orthogonal Greedy Algorthm. OGA) The Orthogonal Greedy Algorthm. For x H we defne G o n = G o nx), for each n N 0, by nducton. G o 0 = 0 and assumng that Go 0, Go 1... Go n 1, and vectors z 1... z n 1 have been defned for some n N we proceed as follows: 1) Choose z n D so that x G o n 1, z n = sup x G o n 1, z z D be the best ap- 2) Defne Z n = spanz 1, z 2... z n )) and let G o n 1 proxmaton of x to Z n,.e. G o n 1 x = nf { z x : z Z n },

30 30 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE denotes the or- whch means that G n 1 = P Zn x), where P Zn thonormal proecton of H onto Z n. GAR) The Greedy Algorthm wth free Relaxaton. For x H we defne G r n = G r nx), for each n N 0, by nducton. G r 0 = 0 and assumng that Gr 0, Gr 1... Gr n 1, have been defned for some n N we proceed as follows: 1) Choose z n D so that x G r n 1, z n = sup x G r n 1, z z D 2) Put G r n = a n G r n 1 + b nz n, where G r n s best approxmaton of x by an element of the two dmensonal space spang r n 1, z n). GAFR) The Greedy Algorthm wth fxed Relaxaton. Let c > 0. For x H we defne G f n = G f nx), for each n N 0, by nducton. G f 0 = 0 and assumng that Gf 0, Gf 1... Gf n 1, have been defned for some n N we proceed as follows: 1) Choose z n D so that x G f n 1, z n = sup x G f n 1, z z D ) 2) Put G f n = c 1 1 n G f n 1 + c n z n, Smlar to the weak purely greedy algorthm there are also weak versons of the orthogonal greedy algorthm and the pure greedy algorthm wth relaxaton and We denote them by WOGA, WGAR and WGAFR. 2.2 Convergence Proposton Assume that we consder the WPGA, WOGA or WGAR and assume for the weakness factors t n ) that 2.2) t 2 k =. k N For x we let x n = x G n x), x n = x G o nx) or x n = x G r x), respectvely. If the sequence x n ) converges t converges to 0.

31 2.2. CONVERGENCE 31 Proof. Assume that x n converges to some u H and u 0. Then, snce D s a dctonary, there s a d D so that δ = d, u > 0 and thus we fnd a large enough N N so that d, x n δ/2, for all n > N In the case that we consder WPGA we obtan for n N x n+1 2 = x n z n+1, x n z n+1 2 = xn 2 z n+1, x n 2 x n 2 t 2 n+1δ 2 /4. and thus for k = 1, 2, 3... x N 2 x N+k 2 = N+k 1 =N x 2 x +1 2 N+k =N But ths s a contradcton. In the case of the WOGA we smlarly have for n N t 2 +1δ 2 /4 N. { n+1 x n+1 2 x = mn a z : a1, a 2,... a n+1 R } x n z n+1, x n z n+1 2 = x n 2 z n+1, x n 2 t 2 n+1δ 2 /4 and we obtan a contradcton as n the WPGA case. Smlarly n the case we consder the WGAR we estmate: { x n+1 2 x = mn ag r } n x) bz n+1 : a, b R x G r nx) z n+1, x n z n+1 2 = x n 2 z n+1, x n 2 x n 2 t 2 n+1δ 2 /4. Theorem Assume that condton 2.2) of Proposton holds. Then G o nx) : n N) as defned n WOGA) converges for all x H to x. Proof. Let x H. For n N let Z n be the space defned n WOGA. G o nx) = P Zn x). Snce Z 1 Z 2 Z 3... t follows that G o nx) converges to P Z x), where Z = n N Z n. Thus the clam follows from Proposton Theorem Assume that the sequence t k ) 0, 1) satsfes 2.3) t k k =. For x X consder the WPGA G n x)) wth weakness factors t n ). Then G n x)) converges. k N

32 32 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE Remark. Snce by Hölder s nequalty t k k k=1 k=1 condton 2.3 mples that k N t2 k =. We wll need the followng Lemma frst t 2 k ) 1/2 k=1 1 ) 1/2, k Lemma Assume y = y ) l 2 and t k ) 0, 1) satsfes 2.3). Then lm nf n y n t n n y = 0. Proof. an alternate, and shorter proof due to Sheng Zhang wll be gven below) We wll prove the followng clam: Clam. If f L 2 [0, ] and we defne then 2.4) 0 F x) = x 0 F 2 x) x 2 dx 4 ft) dt 0 f 2 x) dx. If we apply the clam to the functon f ) = 1 1,] y, t follows that [ 1 n n=1 n ] 2 [ n 1 1 ] 2 y y + y 2 n 4 n=1 0 0 [ 1 x x 0 n=1 ft) dt] 2 dx + n=1 y 2 n=1 f 2 t) dt + y 2 = 5 y 2 It follows therefore from the Cauchy Schwarz nequalty that n=1,t n 0 t n n y n t n n y y n 1 n n=1 n=1 [ n ] 1/2 [ [ 1 y y 2 n n=1 n=1 ] 1/2 n 2 y ] < snce t n n = n=1

33 2.2. CONVERGENCE 33 t follows that lm nf n y n t n n y = 0. In order to prove the clam we can assume that fx) s a postve functon, we note frst that by Hölder s nequalty, and thus F x) = x 0 x ft) dt x 1/2 f 2 t) dt, 0 2.5) F x) x 1/2 x 0 f 2 t) dt x 0 0 For a fxed x 0 > 0 we also deduce from Hölder s nequalty for x > x 0 that F x) F x 0 ) = and thus x x 0 ft) dt x x 0 ) 1/2 F x) x 1/2 F x 0) x 1/2 + x 0 x x 0 f 2 t) dt x 1/2 f 2 t) dt. x 0 f 2 t) dt, By choosng for a gven ε > 0 x 0 far enough out so that x 0 f 2 t) dt < ε/2 and then x 1 > x 0 so that x 1/2 1 F x 0 ) < ε/2, t follows that and thus F x) x 1/2 < ε whenever x > x 1, 2.6) F x) x 1/2 x 0 f 2 t) dt x 0. Usng ntegraton by parts, t follows for any 0 < a < b < that b a F 2 x) x 2 dx = F 2 x) b b x + 2 F x)fx)x 1 dx x=a a [ ] 2 [ ] 2 F a) F b) b a 1/2 + b 1/2 + 2 [By Hölder s nequalty] a ) 1/2 ) 1/2 F 2 x) b x 2 f 2 x) dx a

34 34 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE and thus, n case that a s chosen small enough and b large enough so that F x) does not a.e. vansh on [a, b], we have b a ) 1/2 [ ] F 2 x) [F a) ] 2+ [ F b) ] 2 b x 2 a 1/2 b 1/2 a Our clam follows now by lettng a 0 and b ) 1/2 1/2 F 2 x) b x 2 +2 f 2 x) dx). a Proof by Sheng Zhang. Suppose, to the contrary that and, thus for some n 0 N y n t n δ = lm nf n y n t n n y + > 0, n y + > δ/2 whenever n n 0. For n n 0, we deduce fro Hölders s nequalty δ 2 < 1 t n n y n y 1 n y 2) n y n 2, t n and thus whch yelds lm nf n y n t nn δ 1 2 n y 2, y n t nn δ 2 1 y2 =: ε Thus there s an n > n 0, so that for all n n 1, y n 2 εt n /2n. contradcts the assumpton that y = y n ) l 2 and n=1 t n/n =. But ths Proof of Theorem Let x H and put for n N, G n = G n x) wth where z n D satsfes G n x) = n x G 1, z z, 2.7) z n, x G n 1 = z n, x n 1 t n sup z, x n 1. z D

35 2.2. CONVERGENCE 35 Defne 2.8) x n = x G n x) = x n z, x G 1 z = x n 1 z n, x G n 1 z n, By nducton we show that for every n N 2.9) x n 2 = x 2 n z, x 1 2. Indeed, for n = 1 the clam s clear and assumng that 2.9) s true for n N we compute x n+1 2 = x n x n, z n+1 z n+1 2 n+1 = x n 2 x n, z n+1 z n+1 2 = x 2 z, x 1 2. It follows therefore from 2.9) that 2.10) z, x 1 2 x 2 For m < n we compute 2.11) x n x m 2 = x m 2 x n 2 2 x m x n, x n, and x m x n, x n = = n =m+1 n =m+1 n =m+1 zn+1, x n t n+1 x 1 x, x n x 1 x, x n z, x n z, x 1 d D n =m+1 z, x 1 [ z, x n max d, x n t 1 z n+1, x n ] n+1 z n+1, x n n+1 z, x 1. t n+1

36 36 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE We can therefore apply Lemma to t n ) and y n = z n+1, x n, for n N, and deduce that lm nf x m x n, x n = 0. max n m<n Together wth the fact that x n s decreasng and 2.11) ths mples that there s subsequence x nk ) whch converges to some x H. We clam that the whole sequence x n ) converges to that x, whch, together wth Proposton 2.2.1, would fnsh the proof. Note that for any n N and any k N so that n k > n we have x n x x n x nk + x nk x = x n 2 x nk 2 2 x n x nk, x nk ) 1/2 + xnk x x n 2 x nk 2) 1/2 + 2 max m n k x m x nk, x nk 1/2 + x nk x. So, gven ε > 0 we can choose n 0 large enough so that x n 2 x nk 2) 1/2 < ε/3, for all n n 0 and k wth n k > n. Then we choose k 0 so that for all k > k 0, 2 max m nk x m x nk, x nk 1/2 < ε/3 and x nk x < ε/3. For any n n 0, we can therefore choose k k 0 so that also n k > n, and from above nequaltes we deduce that x n x < ε. The next Theorem proves that at least among the decreasng weakness factors τ the condton 2.3 s optmal n order to mply convergence of the WPGA. Theorem In the class of monotone decreasng sequences τ = t k ), the condton 2.3) s necessary for the WPGA to converge. In other words, f t n ) s a decreasng sequence for whch 2.12) t n n < n N then there s a dctonary D of H, an x H and sequences G n ) H and z n ) D, wth G 0 = 0 so that for x n = x G n the followng s satsfed: 2.13) 2.14) x n = x n 1 x n 1, z n z n x n 1, z n t n max z D x n 1, z, but so that x n does not converge to 0. We wll need the followng notaton: Defnton Assume D S H has the property that z D mples that z D and assume that τ = t n ) N n=1 0, 1], wth N N { } sa fnte sequence of postve numbers. A par of sequences x n ) N n=0 H and z n) N n=0 D

37 2.2. CONVERGENCE 37 and are called a par of WPGA-sequences wth weakness factor τ and dctonary D f x 0 spand ) and for all n = 1, 2... N 2.15) 2.16) x n = x n 1 x n 1, z n z n x n 1, z n t n max z D x n 1, z. Remark. To gven sequence τ = t n ) n=1 0, 1], satsfyng 2.12) we wll choose elements of a dctonary D as well as the elements x n and z n of a par of WPGAsequences wth weakness factor τ and dctonary D recursvely. To acheve that we wll choose nductvely elements x n, n 0 and z n, n 1, so that for all n N 2.17) 2.18) 2.19) x n = x n 1 x n 1, z n x n 1, z n t n max sup x n 1, e, max x n 1, z ) N,2,...n 1 x, z +1 t x, z n, for all = 0, 1, 2... n 1. Here e ) denotes an orthonormal bass of H. We deduce then that x n ) n=0 and z n) n=0 s a par of WPGA-sequences wth weakness factor τ and dctonary D = {±e, ±z : N}. Proof of Theorem The followng procedure s the key observaton towards nductvely producng our example. We let e ) be an orthonormal bass of H. For gven t 0, 1/3] and n N. We defne elements x n spane, e ), n 0 and z n e, e ), z n = 1, n, and α n [0, 1] recursvely untl we stop at some n = N, when some crterum s satsfed, as follows: We put x 0 = e, Now assume that for some n N, we defned x s = a s e + b s e and α s 0, 1) and z s S H for all 1 s n 1 so that for all 1 s < n we have 2.20) 2.21) 2.22) 2.23) x s 1, z s = t, as long as s N 1, z s = α s e 1 αs) 2 1/2 e, a s, b s 0, and a s b s 2, as long as s N, x s = x s 1 x s 1, z s z s. Condtons 2.20) and 2.22) become vacuous once we defned N for s = N) Then we frst defne z n as z n = α n e 1 α 2 n) 1/2 e where α n s defned so that x n 1, z n = t. Secondly defne x n to be x n = x n 1 x n 1, z n z n

38 38 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE and wrte x n as x n = ã n e + b n e. Case 1. ã n b n 2t In that case we choose α n = α n and z n = α n e 1 α 2 n) 1/2 e. Thus 2.20) and 2.21) are satsfed for s = n. Then we let x n = x n, and have therefore satsfed 2.22) and 2.23). Case 2. ã n b n < 2t. Then we let N = N t = n and put α N = 1/ 2, z N = e e )/ 2 and x N = x N 1 x N 1, z N z N. Then 2.21),and 2.23) are satsfed whle 2.20) s vacuous. From the defnton of X N n Case 2, we observe that 2.24) 2.25) x N 1, z N = 2 1/2 a N 1 b N 1 ) t, and t follows therefore that a N = b N = 1 2 a N 1 + b N 1 ). In partcular also 2.22) s satsfed for n = N, assumng that N s fnte, whch we wll see later here the second part of 2.22) s vacuous). Once the second case happens we fnsh the defnton of our sequences. We stll wll have to show that eventual Case 2 wll happens and that N s fnte; for the moment we thnk of N beng an element of N { } We make the followng observatons. From 2.20) and 2.21) we deduce that 2.26) a n+1 = a n tα n+1 and b n+1 = b n + t1 α 2 n+1) 1/2, f n < N 1 whch mples that 2.27) a n+1 b n+1 = a n b n t α n αn+1) 2 1/2) { a n b n 2t f n < N 1. a n b n t Ths yelds N 2 1 = a 0 b 0 a s b s ) a s+1 b s+1 ) N 1)t s=0 and therefore we showed that N s fnte. Snce by defnton of N and ã N and bn 2t > ã N b N t follows that = a N 1 b N 1 t α N + 1 α 2 N) ) 1/2 an 1 b N 1 t ) x N 1, z N = 2 1/2 a N 1 b N 1 ) { 2t t.

39 2.2. CONVERGENCE 39 It follows therefore from 2.27),2.24) and 2.25) that { N 1 tn 1 = a 0 b 0 = a s b s ) a s+1 b s+1 ) 2tN s=0 and thus 2.29) 1 2t N 1 t. From the defnton of x N and 2.28) we deduce that x N 2 = x N 1 2 x N 1, z N 2 By 2.28)) x N 1 2 4t 2 = x and thus, snce t 1/3, N 1 s=1 xs 2 x s 1 2) 4t 2 = x 0 2 N 1)t 2 4t 2 x 0 2 t 3t 2 By 2.29) 2.30) x N 2 x 0 2 2t Fnally note that the sequence x n ) N n=0 s a WAGD sequence for the Dctonary D = {z n : n = 1, 2... N t } {e } wth the weakness factor t. We call x n ) Nt n=0 together wth the sequence z n ) n=0 Nt the WAGD sequence generated by t and the par e, e ). Now we assume that t n ) n=1 s a sequence n 0, 1], so that t n <. We frst requre the addtonal assumpton that tn n < < ε = It follows that t 2 s = t 1 + t 2 + t 4 + t 8... s=0 t 1 + t t 3 + t 4 ) t 5 + t 6 + t 7 + t 8 ) t 9 + t t 16 ) +... [ 2 t 1 + t t 3 + t 4 ) t 5 + t 6 + t 7 + t 8 ) + 1 ] 16 t 9 + t t 16 ) +... t n 2 n 2ε. n=1 We wll construct recursvely sequences x n : n = 0, 1, 2,...) and z n : n = 1, 2...) so that x 0 = e 1, x n = x n 1 x n 1, z n z n, so that for every n N 2.31) x n 1, z n t n max x n 1, z and x n 1, z n t n sup x n 1, e,...n 1 N

40 40 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE and 2.32) t x, z n x, z +1 for all = 0, 1, 2,... n 1. As noted n the remark before the proof, these two condtons wll ensure that that for each n the vector s of the form x n = x G n x), where G n x) : n N 0 ) s the result of a WPGA wth weakness factors t n ) and dctonary D = {z n, e n : n N}. We start wth x = x 0 = e 1, and let x 1) n : n = 0, 1, 2,... N t1 ) and z 1) n : n = 1, 2,... N t1 ) be the WAGD sequence generated by t and the par e 1, e 2 ), then we put x n = x 1) n and z n = z 1) n for n = 1, 2, 3... N t1. Note that we satsfed so far our requred condtons 2.31) and 2.32) snce by constructon x n 1, z n = t 1 t n t n x n 1 for all n = 1, 2..., N 1 = N t1. By 2.25) x N1 s of the form x N1 = c 1 e 1 e 2 ), and we deduce from 2.30) and the fact that x N1 1 that c 2 1 1/2, N 1 1 and x N t 1. Then we consder let x 2,1) n : n = 0, 1,... N t2 ) and z n 2,1) : n = 1, 2... N t2 ) be the WAGD sequence generated by t 2 and the par e 1, e 3 ), and x 2,2) n : n = 0, 1,... N t2 ) and z n 2,2) : n = 1, 2... N t2 ) be the WAGD sequence generated by t 2 and the par e 2, e 4 ). We put N 2 = N t2 and for n = 1, 2,... N 2 we defne x N1 +n = c 1 x 2,1) n + c 1 e 2 and z N1 +n = z n 2,1) x N1 +N 2 +n = c 1 x 2,1) N 1 + c 1 x 2,2) n and z N1 +N 2 +n = z n 2,2). We observe that for n = 1, 2... N 2 x N1 +n 1, z N1 +n = c 1 t 2 t 2 max s N x N 1 +n 1, e s and x N1 +n 1, z N1 +n = c 1 t 2 t 2 max s=1,2,...n 1 x N1 +n 1, z s the frst nequalty follows from the fact that the coordnates of x N1 +n, n = 1, 2... N 2 are absolutely, not larger than c 1, the second nequalty follows from 2.21) and the fact moreover the coordnates of x N1 +n, n = 1, 2... N 2 are not negatve whle z s, s = 1, 2... N 1, has a postve and negatve coordnate). Secondly we note that for = 1, 2,..., N 1 and n = 1, 2,..., N 2 1, t follows from 2.20) and 2.24) that t x, z N1 +n t 1 x, z +1. Ths mples that the condtons 2.31) and 2.32) hold for all N 1 n N 1 + N 2. Smlarly we can show that they also hold for all N 1 + N 2 n N 1 + 2N 2. Fnally 2.30) mples that X N1 +2N 2 s of the form X N1 +2N 2 = c 2 e 1 + e 2 + e 3 + e 4 )

41 2.2. CONVERGENCE 41 wth c 2 2 1/4 and x N1 +2N 2 2 x N1 2 c 2 12t 2 c 2 12t 2 1 2t 1 2t 2. Now assume that for some r N we have chosen x n : n = 1, 2,... M r ), wth M r = and r 2 1 N and N = N t2 1, = 1, 2... r, z n : n = 1, 2,... M r ) so that 2.31) and 2.32) hold for all n M r, and so that for some c r wth c 2 r 2 r, and so that 2r x Mr = c r =1 e x Mr 2 1 2t 1 2t 2 2t t 2 r 1 then we let for = 1, r x r+1,) n : n = 0,... N r+1 ) and z r+1,) n : n = 0,... N r+1 ),wth N r+1 = N t2 r, be the WPGA sequences generated by by t 2 r and the par e, e 2 r +), and fnally put for = 1, r and n = 1, 2,... N r 1 x Mr+ 1)Nr+1 +n = c r x r+1,s) N r s=1 z Mr+ 1)N r+1 +n = z r+1,) n. + c r x r+1,) n + c r 2 r s=+1 e s, and We deduce as n the case r = 1 that the condtons 2.31) and 2.32) hold for all n M r + 2 r N r+1 = r+1 s=1 2s 1 N s = M r+1, that x Mr+1 = c 2 r+1 r+1 s=1 e s, for some c r+1 2 r 1, and that x Mr+1 1 2t 1 2t 2,... 2t r. Ths fnshes the choce of the the x n and z n. Snce x Mr r s=1 t 12 2s 1 4ε > 1 16 = 1 4, t follows that x n) does not converge. We therefore proved our clam under the addtonal assumpton that n=1 t n/n) < 3/16. In the general case we proceed as follows. We frst fnd an n 0 so that s=n 0 t 2 s < 3/16,

42 42 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE and let 2 n 0 x = e. Then we choose z = e, = 1, 2..., 2 n 0 1, and thus x 0 = x and recursvely x n = x n 1 x n 1, z n = = n + 1 2n0 e for n = 1, 2,... 2 n 0 1. In partcular x 2n0 1 = e 2 n 0 from then on we choose x 2n0 1+n = x n, n = 1, 2... and z 2n0 1+n = z n, where the x n and z n are chosen lke the x n and the z n n the specal case, but n the Hlbertspace H = spane : 2 n 0 ). 2.3 Convergence Rates Note that wthout any specal condtons on the startng pont n the Pure greedy algorthm or others) we can not expect beng able to estmate the convergence rate. Indeed let ξ n ) be any sequence of postve numbers, whch decreases to 0 and let D = {±e n : n N}, where e n ) s an orthonormal bass of our Hlbert space H, then take x = ξ ξ +1 e then t follows for the n-th approxmates G n = G n x) defne as n PGA) and thus G n = x G n 2 = n ξ ξ +1 e +n ξ ξ +1 = ξ n+1. Thus no matter how slow ξ n ) converges to 0, there s a x so that G n x) converges at least as slow as ξ n ). In order to state our frst result we ntroduce for a dctonary D of H the followng lnear subspace: { } 2.33) A 1 = A 1 D) = c z <. z D c z z : c z ) K and z D For x A 1 we put { 2.34) x A1 = nf c z : c z ) K and f = c z }. z D z D

43 2.3. CONVERGENCE RATES 43 Theorem [DT] Assume D s a dctonary of a separable Hlbert space. Let x A 1 D) and assume that G n ) = G n x)) s defned as n PGA) and let x n = x G n, for n N. Then 2.35) x n f A1 n 1/6 for n N. For the proof of Theorem we need the followng observaton. Lemma Assumng that ξ m ) s a sequence of postve numbers so that for some number A > ) ξ 1 A and ξ m+1 ξ m 1 ξ m /A), for m 1. Then 2.37) ξ m A, for all m N. m Proof. We assume A = 1 pass to ξ m = ξ m /A) We prove the clam by nducton for each m N. For m = ) follows from the assumpton. Assume that the clam s true for m N. If ξ m 1 m+1 then also ξ m+1 1 m+1 snce from 2.36) t 1 follows that the sequence ξ ) s decreasng. If m+1 < ξ m 1 m we deduce that ξ m+1 ξ m 1 ξ m ) 1 m 1 1 ) m + 1 = 1 m m m + 1 = 1 m + 1, whch mples the clam for m + 1 and fnshes the nducton step. Proof of Theorem For x H we put x, z ρx) = sup z D x. Note that f x A 1, η > 0 and c z ) z D R + 0 s such that x = z D c zz and cz η + x A1 t follows that x 2 = x, c z z c z sup z, x x A1 + η ) x ρx), z D z D z D and, thus, snce η > 0 was arbtrary, 2.38) ρx) x x A1. Let x A 1 and let us assume that there s a representaton x = z D c zz so that x A1 = z D c z otherwse we use arbtrary approxmatons). Let z m )

44 44 CHAPTER 2. GREEDY ALGORITHMS IN HILBERT SPACE and G m ) be defned as n PGA) and x n = x G n, for n N. We note that for m N 0 xm+1 2 = xm xm, zm+1 zm ) = x m 2 x m, z m+1 2 = x m 2 1 ρ 2 x m )). Puttng a m = x m 2, b 0 = x 0 A1 = x A1 and, assumng that b m has been defned, we let b m+1 = b m + ρx m ) x m = b m + ρx m )a 1/2 m. Frst we observe that 2.40) x m A1 b m. Indeed, for m = 0 ths smply follows from the defnton of b 0, and assumng 2.40) holds for m N 0 t follows that x m+1 A1 = x m x m, z m+1 z m+1 A1 x m A1 + x m, z m+1 = x m A1 + ρx m ) x m = b m+1. Secondly we compute usng 2.39), 2.38) and 2.40) a m+1 = x m+1 2 = a m 1 ρ 2 x m )) a m 1 x m 2 ) x m 2 a m 1 a ) m A 1 b 2 m and thus, snce b m+1 b m a m+1 b 2 m+1 a m+1 b 2 m a m b 2 m 1 a m b 2 m ). Note that a 0 b 2 0 ξ n = a n 1 b 2 n 1 = x 2 x 2 A 1 and deduce that 1. We therefore apply Lemma to sequence ξ n wth 2.41) a m b 2 m 1, whenever m N. m Snce by the recursve defnton of b ), 2.40) and 2.38) we get b m+1 = b m 1 + ρxm )a 1/2 m b 1 ) m bm 1 + ρxm )a 1/2 m x m 1 ) A 1 bm 1 + ρ 2 x m ) ), we obtan together wth 2.39) a m+1 b m+1 a m b m 1 ρ 2 x m ))1 + ρ 2 x m )) a m b m. a m b m ) s therefore decreasng and a m b m a 0 b 0 = x 2 x A1. Multplyng both sdes of 2.41) by a 2 mb 2 m we obtan therfore a 3 m a2 mb 2 m m x 4 x 2 A 1, m whch mples our clam after takng on both sdes the sxth root.

45 2.3. CONVERGENCE RATES 45 The next Example due to DeVore and Temlyakov gves a lower bound for the convergence rate of PGA) Example [DT] Let H be a separable Hlbertspace and h ) an orthonormal bass of H. We wll defne a dctonary D H, a vector x H for whch x A1 D) = 2, and so that Defne and Note that x m = x G m a = ) 23 1/2 and A = 11 z = Ah 1 + h 2 ) + aa c m, for m N. ) 33 1/2 89 ) 1/2hk kk + 1). k=3 z 2 = 2A 2 + a 2 A 2 k=3 = 2A 2 + a 2 A 2 k=3 = 2A a2 A 2 = k k k 1 k ) = 1. Put D = {±g} {±h : N} and let x = h 1 + h 2 and we apply PGA) to f Clam: In Step 1 of PGA) we have z 1 = z and x 1 = x x, z z = 1 2A 2 )h 1 + h 2 ) 2aA 2 ) 1/2hk kk + 1). Indeed, Thus z 1 = z and x, z = 2A > 1, x, h 1 = x, h 2 = A, and x, h = 0, f, > 2. k=3 x 1 = x x, z z ) ) 1/2hk = h 1 + h 2 2A Ah 1 + h 2 ) + aa kk + 1) k=3