MP IN BLOCK QUASI-INCOHERENT DICTIONARIES

Transcription

1 CHOOL O ENGINEERING - TI IGNAL PROCEING INTITUTE Lornzo Potta and Prr Vandrghynst CH-1015 LAUANNE Tlphon: Tlfax: mal: lornzo.potta@pfl.ch ÉCOLE POLYTECHNIQUE ÉDÉRALE DE LAUANNE P IN BLOCK QUAI-INCOHERENT DICTIONARIE Lornzo Potta and Prr Vandrghynst wss dral Insttut of Tchnology Lausann EPL) gnal Procssng Insttut Tchncal Rport TR-IT Dcmbr 15th, 2003

2 Œ : 2 P n Block Quas-Incohrnt Dctonars Lornzo Potta and Prr Vandrghynst gnal Procssng Insttut wss dral Insttut of Tchnology Lausann, wtzrland I. BLOCK INCOHERENT DICTIONARIE Gvn a rdundant dctonary, w consdr th followng -subst dcomposton, and w call blocks th substs of atoms,. Th block cohrnc s dfnd as th maxmum cohrnc btwn any two atoms, takn from dffrnt blocks. Dfnton 1: Th block cohrnc, gvn a block dcomposton, s! #"%$'& *), "-$.& / !6 / ) whr 6 / s th 95:<; atom from th block. Dfnton 2: A dctonary s thn sad block ncohrnt f thr xsts a dcomposton such that th block cohrnc = s small. Th block cohrnc consdrs smlarts btwn atoms from two dffrnt block. In ordr to rfn th analyss of th cohrnc, w ntroduc anothr functon, calld ED.9 functon, that rprsnts th cohrnc btwn sts of G blocks IH J <K H, wth L >N.OQPERN TG. Dfnton 3: Lt UV functon s Dfnton 4: A gvn dctonary W dnot a dcomposton, and XH U YK H rprsnt a st of G [Z P G H 0 \]"%$'& ]^ H ^ W "-$.& ` K H 0 "-$'& ba<k H /dc254!6 /dc 6 blocks. ED ) s sad to b block quas-ncohrnt, f w can fnd a block dcomposton such that Z P G grows slowly wth G. Th block cohrnc consdrs cohrnc btwn two blocks, and th Babl block functon Z P G masurs cohrnc btwn G blocks. Notc that ED.9 functon s boundd by <D.9fD'g B1bB1h B : Z P G j GkW. Th dfntons of th prvous functons s th xtnson of th cohrnc and th V>A@CB functon P G ntroducd by Donoho, Huo, Elad [1], [2]. Th?>A@CB functon was dvlopd and utlzd by Troop n th Exact Rcovry Thorm [3]. W nd now also to consdr th cohrnc wthn a sngl block. Gnrally, a sngl block has a strong cohrnc.., th Babl functon grows quckly). or a mor dtald analyss, w ar howvr ntrstd n a functon that rprsnt th cohrnc of a partcular subst of functons n, and w call t th D.B functon l PE. Dfnton 5: Th D.B functon rlatd to a block s l P< p 0 \] ]^ "%mon : Wq ^ rsctu/1v cyw "-$.& / a / ) 254!6 6 / 7 2 3) whr p s a st of ndpndnt atoms from,.., pyx and z { >Nh=P< p #z { >uh=pe. Th D.B functon l P< ndcats how much th atoms n a block can spak dffrnt languags. In othr words, t llustrats how clos a bass constructd wth atoms from block s to an orthogonal bass that spans th rang } P<. Th st of atoms,.. p, whr th Bor functon s mnmal, s calld ~. If l P<, w can fnd a st o~ x that s an orthogonal bass for z { >uh=pe. Th xtnson of th D bb functon to th dctonary s fnally dfnd as l P "-$.& l PE. II. EXACT BLOCK ELECTION Usng th dfntons dfnd n c. I, w prov n ths scton that, gvn a block ncohrnt dctonary and a sgnal, th atchng Pursut P) algorthm can rcovr a block-spars rprsntaton of. W consdr hr th rstrctd problm P ƒh. -PARE, whch mans that s a lnar combnaton of atoms blongng to a subst of G blocks,?h <K H. rstly, w fnd a sngl suffcnt condton undr whch atchng Pursut rcovrs atoms from a gvn st of ncohrnt blocks H. In ths cas, w say that P chooss atoms from corrct blocks R. Lt rprsnt H as an oprator or matrx, and lt Vˆ H dnot ts psudonvrs. Thorm 1: Lt a block ncohrnt dctonary and H <K H. If th sgnal Š H z { >Nh=PE H, undr th rcovry condton PEƒHbU"-$'& K` ˆ Ž H 6 4 4)

3 Œ 3 thn w hav that P: 1) pcks up atoms only from corrct blocks = R, 2) convrgs xponntally to f. Proof of Thorm 1. W follow th proof for Exact Rcovry thorm [3]. uppos that t Š H. If an atom 6 t from H s slctd by th atchng Pursut algorthm, thn t t 4 6 t t 7 6 t blongs to H, wth. Th vctor H t lsts th nnr products btwn th rsdual t and all th atoms from th blocks, 8 R ; takng th norm of ths vctor w hav that H t s th largst of ths nnr products n magntud, whr H rprsnts th complx conjugat of H. Th numbr H t corrsponds to th largst nnr product n magntud btwn t and an atom that dos not blong to XH, that mans 6 H. An atom s slctd from th corrct block, = R, whn th followng quotnt s lss than on PY t H t H t 4 5) By assumpton, t H, and H?ˆ H s a projctor onto th rang of H. Thrfor, usng th proprts of th psudonvrs, t PE Hˆ H t and P< t H P<Vˆ H H t H t whr th matrx norm 0 H P<Vˆ H s th norm nducd by th vctor norm P< t H P<Vˆ H Vˆ H H "%$'& K Ž 0 Vˆ H 6 0. Usng proprts of th matrx norm w obtan so PY t j PEƒHb 4 whch mans that P slcts an atom from XH. By nducton th frst part s provd. To prov th scond part, w just notc that P s facd wth a fnt dmntonal spac H, and w know that P n a fnt dmntonal spac s xponntally convrgnt. As a corollary, th followng thorm gvs a condton undr whch rght block slcton s n forc whn blongs to th ncohrnt blocks. z[{ >Nh of an arbtrary st of G Thorm 2: Lt a block ncohrnt dctonary and H an arbtrary st of G blocks and J "-$'&.>Nh 9 P<. If th sgnal Š H and 3 Z P G l P 3 Z P G 4 6) thn w hav that P: 1) pcks up atoms only from th corrct blocks, 2) convrgs xponntally to f. Proof of Thorm 2. Th proof s agan gvn by nducton. W suppos that t H. If an atom from IH s slctd, thn t H. W ndcat wth H YK H ~ th unon of th G sts assocatd to th G blocks n th dfnton 5) of th D.B functon. Now w dfn XH ~ to b a st of lnar ndpndnt atoms from H such that 2 IH ~ 2.>Nh 9 P< H. It follows that z { >Nh=PEIH ~ Jz[{ >Nh=P<IH1 H, ƒh ~ s a bass for H, thrfor t P< H ˆ ~b H ~ t and P< t H t H t H P< H ˆ ~ H ~ t H t

4 / 4 snc IH ~ x ƒh w hav that H t H ~ t P< t and "%$'& K Ž H P<Vˆ H ~1 H ~ t H ~ t H P< H ˆ ~ Now w can xpand th psudonvrs and apply th norm bound PY t "-$'& K Ž P< PE H ~ H ~ 0 H ˆ ~ 6 H ~ H ~ H ~ 6 "-$.& K Ž W can asly bound th scond trm of th rght part of 7) usng th ED.9 functon "-$.& K Ž H ~ 6 "-$.& K Ž a K Ž ~ H ~ )! Z P G 8) whr J "%$'&.>uh 9 P<. In ordr to bound th frst trm of th rght part of 7) w us th Von Numann srs to comput th nvrs P< H ~ H ~. Wrtng H ~ H ~ d0 4, t follows that : PE H ~ H ~ d0, whr s th dntty matrx, and undr th condton that P a /C Th matrx has zro dagonal and th valus out of dagonal corrspond to th nnr product btwn atoms from H ~, takng nto account th structur of H ~ t s composd by G ncohrnt blocks) w can bound th norm usng th D bb <D.9 functon: H ~ 6 / d0 d0 "-$.& / a / ) a /f P d0 H ~ / 7 2 and puttng togthr th bounds 8),9) nto 7) w obtan for P< t th bound P< t j 3 Z P G P l P! Z P G [ o undr th condton l P! [Z P G 9)! Z P G l P! Z P G 4 t follows that P< t 4 and P slcts an atom from th corrct block, by nducton th frst part s provd. or th scond part, w ar n th sam condton as n thorm 1. Usng th bound for ED.9 functon [Z P G Z P G G# t follows from Thorm 2 that, f th sgnal blongs to th z { >Nh of G blocks, thn P rcovrs atoms from th corrct blocks whn G l P 4 3W

5 t 5 III. RATE O CONVERGENCE Anothr mportant factor that dtrmns th qualty of a sgnal xpanson s th rat of convrgnc of th approxmaton. Ths rat can also b boundd wth th hlp of th cohrnc dfnd prvously, n th cas of block ncohrnt dctonars. Thorm 3: If th sgnal Š H and Š Z P G l P Š Z P G 4, thn P pcks up atoms only from th corrct blocks at ach stp and t l P! Z P G!G 10) Proof of Thorm 3. rom thorm 2 w know that undr condton 6) P pcks up atoms from th rght st of blocks H. At ach stp th rsdual blongs to th spac H, and th nrgy of th rsdual s t t "-$.& K Ž 2 t 6 2 t "-$.& K Ž 2 t 6 f2 t In ordr to bound th dcay of th rsdual nrgy, w nd a lowr bound for "-$.& K Ž ) wth H z { >uh=pe H z { >Nh=P< H, whr th st H s dfnd n th prof of thorm 2. nc w can wrt lk a combnaton of lmnts from H, H 6, w hav and w obtan th followng lowr bound for 12) "%$'& K Ž a 6 a 2 < "-$.& K Ž "-$.& K Ž ) 13) 14) W wsh to chang wth n ordr to bound 14) wth th mnmum norm of th oprator H. W know that b>nh 9 PE H 1 { IG, whr "-$.&.>Nh 9 PE, wch mans that { IG, and usng th Gnsn nqualty w hav and puttng th uppr bound I { a { 3{ a a { 2 2 { 2 2 IG nto 14) w obtan "-$.& K Ž IG { 15) Usng th Thn ngular Valu Dcomposton H, whr and ar orthogonal whl s dagonal wth full rank snc H has full rank, w can bound th oprator H wrtng P ƒh a t t 16)

6 t t 6 Th squar sngular valus of XH concd to th gnvalus of th th Gram matrx H IH, ndd and ar smlar matrxs. Th gnvalu t t can b bound usng th Gršgorn dsc thorm: vry gnvalu of ls n on of th { dscs Th matrx Dsc/ 2 // 2 a / ) 2 / 2 has untary dagonal snc th normalzaton of th atoms. Takng nto account th block ncohrnt structur of H w can bound th sum abov wth 2 t 2 a / ) 2 / 2 l P ƒ Z P G and th squar mnmum sngular valu t l P I [Z P G. Puttng ths bound nto 16) and 15) w obtan "-$.& K Ž l P I Z P G IG and fnally from 11) w nd th proof t t l P I [Z P G IG l P I Z P G IG Thorm 4: If th sgnal Š H and Š [Z P G l P Š Z P G 4, thn P pcks up atoms only from th corrct blocks at ach stp and t G whr "%m n, and s rlatd to th rdundancy and structur of block [4]. Lmma 1: Lt H z[{ >Nh=P< H wth H W, f w ndcat wth Jz { >Nh=P<, t follows that a 17) th projcton of nto th spac 18) Proof of lmma 1. uppos for smplcty G. Lt O.>uh 9 P< and O H.>Nh 9 PE H, w buld an orthogonal bass for H z { >Nh=P< takng O orthonormal vctors from, w collct thm nto th matrx, and O H O orthonormal vctors from that ar orthogonal to, w collct thm nto. W buld an orthogonal bass for startng from and addng O O O H orthonormal vctors from that ar orthogonal to, whch w collct nto. Wth ths notaton w 7 : and rspct th bass that w hav that s an orthogonal bass for, 2 for and 2 for H. W can gnralz ths procdur to G W z[{ >Nh=P 2 and H z { >uh=p 2 2. It s now asy to proof 18), wrtng dfnd abov th nrgy of projctd nto s > a 2 > > 2 2 > >. >! ", and w can conclud > a a a > >

7 G t G / 7 Proof of Thorm 4. By nducton w know that th squnc of rsduals t H. Th normalzaton of th atoms mpls t t "-$.& K Ž 2 t ) In ordr to caractrz th dcay of th rsdual nrgy, w nd a manngfull lowr bound for "-$.& K Ž Š H. If P slcts an atom from th block, t foolows that "%$'& K Ž "-$.& K whr "mon P and s th structural rdundancy factor of th block [4]. Inqualty 20) can b drvd analzng th cas of rsdual wth nrgy unformly sprad n all spacs, that mans for all R. Usng lmma 1 t follows that Whn th nrgy s not unformly sprad, t mans thr s at last on componnt and P wll slcts an atom from /. Puttng 20) n 19) w nd th proof t t t 6 whr 20) G 21) G, 9 R, wth nrgy bggr than 21) nc th dmnson of th vctor spacs gnratd by z { >Nh=P< s supposd to b small, w xpct to b clos to on. Th trm G that dvds could b substtutd, takng nto account th block ncohrnt structur of th dctonary. If w hav clos to on, and G rplacd by g P G G, w thus prov th good approxmaton bhavor of atchng Pursut for structurd sgnals, that w obsrv on xprmntal rsults. W clam that takng track of th blocks slctd, du to th block quas-ncohrnt structur of th dctonary, th nrgy bound 21) can b rfnd. Hr thr s som argumnts for th cas of H #z { >Nh=P<IHb wth IH. Th nrgy of th rsdual aftr two traton s boundd by wth "%mon P. REERENCE [1] Donoho D.L. and Huo, X., Uncrtanty prncpls and dal atom dcomposton, IEEE Trans. Inform. Thory, vol. 47, no. 7, pp , Nov [2] Elad. and Bruckstn A.., A gnralzd uncrtanty prncpls and spars rprsntaton n pars of bass, IEEE Trans. Inform. Thory, vol. 48, no. 9, pp , p [3] Tropp, J., Grd s good : Algorthmc rsults for spars approxmaton, Txas Insttut for Computatonal Engnrng and cncs, Tch. Rp., [4] rossard P. and Vandrghynst P., Rdundancy n Non-Orthogonal Transform, n Procdngs of IEEE Intrnatonal ymposum on Informaton Thory, Jun P