Supplementary Material for Fast and Provable Algorithms for Spectrally Sparse Signal Reconstruction via Low-Rank Hankel Matrix Completion

Transcription

1 Suppleentary Material for Fast and Provable Algoriths for Spectrally Sparse Signal Reconstruction via Low-Ran Hanel Matrix Copletion Jian-Feng Cai Tianing Wang Ke Wei March 1, 017 Abstract We establish theoretical recovery guarantees of FIHT for ulti-diensional spectrally sparse signal reconstruction probles, which are straightforward extensions of what we have proved for one-diensional signals in [1]. Assue the underlying ulti-diensional spectrally sparse signal is of odel order r and total diension. We show that Or log nuber of easureents are sufficient for FIHT with resapling initialization to achieve reliable reconstruction provided the signal satisfies the incoherence property. 1 Recovery Guarantees Without loss of generality, we discuss the three-diensional setting. Recall that a three-diensional array X C 1 3 is spectrally sparse if with X l 1, l, l 3 = r d y l 1 z l w l 3, l 1, l, l 3 [ 1 ] [ ] [ 3 ] y = expπıf 1 τ 1, z = expπıf τ, and w = expπıf 3 τ 3 for frequency triples f = f 1, f, f 3 [0, 1 3 and dapling factor triples τ = τ 1, τ, τ 3 R 3 +. Concatenating the coluns of X, we get a signal x of length 1 3. Define = 1 3. We for a three-fold Hanel atrix Hx, which has Vanderonde decoposition in the for Hx = E L DER T, where the -th coluns 1 r of E L and E R are given by E :, L = E :, R = { } y l 1 z l w l 3, l 1, l, l 3 [p 1 ] [p ] [p 3 ], { } y l 1 z l w l 3, l 1, l, l 3 [q 1 ] [q ] [q 3 ], Departent of Matheatics, Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong SAR, China. Eail: jfcai@ust.h Departent of Matheatics, University of Iowa, Iowa City, Iowa, USA. Eail: tianing-wang@uiowa.edu Departent of Matheatics, University of California, Davis, California, USA. Eail: ewei@ath.ucdavis.edu 1

2 where p i + q i = i + 1 for 1 i 3 and D = diagd 1,, d r is a diagonal atrix. It can be verified that if all d s are non-zeros and there exists i, 1 i 3, such that all f i s are distinct, Hx is a ran r atrix. The incoherence property is defined siilarly. Definition 1. The ran r three-fold Hanel atrix Hx with the Vanderonde decoposition Hx = E L DE T R is said to be µ 0-incoherent if there exists a nuerical constant µ 0 > 0 such that σ in E LE L p 1p p 3 µ 0, σ in E RE R q 1q q 3 µ 0. Fro [3, Th. 1], in the undaping case, if the iniu wrap-around distance between the frequencies {f i } r is greater than about i for 1 i 3, this property can be satisfied. Let Hx = UΣV be the reduced SVD of Hx and P U and P V respectively be the orthogonal projections onto the subspaces spanned by U and V. The following lea follows directly fro Def. 1. Lea 1. Let Hx = UΣV = E L DER T. Define c s = ax{ 1 µ 0 incoherent, then U i,: µ 0 c s r P U H a F µ 0c s r and and 3 p 1 p p 3, 1 3 q 1 q q 3 }. Assue Hx is V j,: µ 0 c s r, 1 P V H a F µ 0c s r, where {H a } 1 a=0 fors an orthonoral basis of the three-fold Hanel atrices. Proof. The proof of can be found in []. We include the proof here to be self-contained. We only prove the left inequalities of 1 and as the right ones can be siilarly established. Since U C p 1p p 3 r and E l C p 1p p 3 r spans the sae subspace and U is orthogonal, there exists an orthonoral atrix Q C r r such that U = E L E L E L 1/ Q. So and U i,: = e i E L E LE L 1/ e i E L E LE L 1 P U H a F = UU H a F = E L E LE L 1 E LH a F E L H a F σ in E L E L µ 0r µ 0c s r p 1 p p 3 µ 0r µ 0c s r p 1 p p 3, where we have used the fact that H a has at ost one nonzero eleent in every row and every colun and it only has w a nonzero entries of agnitude 1/ w a and the agnitudes of the entries of E L is bounded above by one for both the daped and undaped case. 1.1 Initialization via One Step Hard Thresholding Our first initial guess is L 0 = p 1 T r HP Ω x, which is obtained by truncating the three-fold Hanel atrix constructed fro observed entries of x. The following lea which is of independent interest bounds the deviation of L 0 fro Hx.

3 Lea. Assue Hx is µ 0 -incoherent. Then there exists a universal constant C > 0 such that µ0 c s r log L 0 Hx C Hx with probability at least 1. The following theoretical recovery guarantee can be established for FIHT based on this lea. Theore 1 Guarantee I. Assue Hx is µ 0 -incoherent. Let 0 < ε 0 < 1 10 be a nuerical constant and ν = 10ε 0 < 1. Then with probability at least 1 3, the iterates generated by FIHT with the initial guess L 0 = p 1 T r HP Ω x satisfy provided x l x ν l L 0 Hx F, { } C ax ε 0 µ 0c s, 1 + ε 0 ε 1 0 µ1/ 0 c 1/ s κr 1/ log 3/ for soe universal constant C > 0, where κ = σaxhx σ in Hx Rear 1. Since Hx = E L DER T, we have κ σ axe L σ in E L ax d in d σaxe R σ in E R. denotes the condition nuber of Hx. It follows fro [3, Th. 1] that σ ax E L resp. σ ax E R and σ in E L resp. σ in E R are both proportional to p 1 p p 3 resp. q 1 q q 3 when the frequencies of x are well separated. Thus the condition nuber of Hx is essentially proportional to the dynaical range ax d / in d. Since the nuber of easureents required in Th. 1 is proportional to c s, it aes sense to set p i to be about the sae as q i for 1 i Initialization via Resapling and Triing To eliinate the dependence on, we investigate another initialization procedure via resapling and triing. The following lea provides an estiation of the approxiation accuracy of the initial guess returned by the Alg. 3. Lea 3. Assue Hx is µ 0 -incoherent. Then with probability at least 1 L+1, the output of Alg. 3 satisfies L 5 L σ in Hx L Hx F 6 56κ provided Cµ 0 c s κ 6 r log for soe universal constant C > 0. We can obtain the following recovery guarantee for FIHT with L 0 being the output of Alg. 3. Theore Guarantee II. Assue Hx is µ 0 -incoherent. Let 0 < ε 0 < and L = log log 16ε 0. Define ν = 10ε 0 < 1. Then with probability at least 1 L + 3, the iterates generated by FIHT with L 0 = L L the output of Alg. 3 satisfies x l x ν l L 0 Hx F, 3

4 provided for soe universal constant C > 0. Proofs Cµ 0 c s κ 6 r log log log We first introduce several new variables and notation. Recall that H is an which aps a vector to a three-fold Hanel atrix and H is the adjoint of H. Moreover, D = H H = diagw 0,, w 1 is a diagonal operator which ultiply the a-th entry of a vector by the nuber of nonzero eleents in H a. Define G = HD 1. Then the adjoint of G is given by G = D 1 H. It can be easily verified that G and G have the following properties: G G = I, G = 1 and G 1; Gz = 1 a=0 z ah a, z C ; G Z = { Z, H a } 1 a=0, Z Cp 1p p 3 q 1 q q 3. 16ε 0 otice that the iteration of FIHT can be written in a copact for x l+1 = H T r P Sl Hx l + p 1 P Ω x x l. 3 So if we define y = Dx and y l = Dx l, the following iteration can be established for y l y l+1 = G T r P Sl Gy l + p 1 P Ω y y l 4 since P Ω and D 1 coute with each other. For ease of exposition, we will prove the leas and theores in ters of y l and y but note that the results in ters of x l and x follow iediately since Hx = Gy and x l x = D 1 y l y yl y. 5 The following suppleentary results fro the literature but using our notation will be used repeatedly in the proofs of the ain results. Lea 4 [4, Proposition 3.3]. Under the sapling with replaceent odel, the axiu nuber of repetitions of any entry in Ω is less than 8 log with probability at least 1 provided 9. Lea 5 [, Lea 3]. Let U C p 1p p 3 r and V C q 1q q 3 r be two orthogonal atrices which satisfy Then P U H a F µc sr and P S GG P S p 1 P S GP Ω G P S holds with probability at least 1 provided that 3µc s r log. 4 P V H a F µc sr. 3µcs r log 6

5 Lea 6 [6, Lea 4.1]. Let L l = U l Σ l Vl be another ran r atrix and S l be the tangent space of the ran r atrix anifold at L l. Then I P Sl L l Gy F L l Gy F σ in Gy, P S l P S L l Gy F σ in Gy. Lea 7 [5, Theore 1.6]. Consider a finite sequence {Z } of independent, rando atrices with diensions d 1 d. Assue that each rando atrix satisfies E Z = 0 and Z R alost surely. Define Then for all t 0, { } σ := ax E Z Z, E Z Z. { } P Z t t / d 1 + d exp σ. + Rt/3.1 Local Convergence We begin with a deterinistic convergence result which characterizes the basin of attraction for FIHT. If the initial guess is located in this attraction region, FIHT will converge linearly to the underlying true solution. Theore 3. Assue 0 < ε 0 < 1 10 and the following conditions P Ω 8 log, 7 P S GG P S p 1 P S GP Ω G P S ε 0, 8 L 0 Gy F σ in Gy p 1/ ε 0 16 log1 + ε 0 9 are satisfied. Then the iterate y l in 4 satisfies y l y ν l L 0 Gy F with ν = 10ε 0 < 1. The proof of Th. 3 aes use of the restricted isoetry property of P Ω on S l when L l is in a sall neighborhood of Gy. Lea 8. Suppose 7, 8 hold and L l Gy F σ in Gy p 1/ ε 0 16 log1 + ε Then we have and P Ω G P Sl 8 log1 + ε 0 p 1/ 11 P Sl GG P Sl p 1 P Sl GP Ω G P Sl 4ε

6 Proof. Since P S GP Ω = P S GP Ω = P Ω G P S, for any Z C p 1p p 3 q 1 q q 3, P Ω G P S Z = P Ω G P S Z, P Ω G P S Z 8 log G P S Z, P Ω G P S Z = 8 log Z, P S GP Ω G P S Z 8 log1 + ε 0 p Z F where the first inequality follows fro 7 and the second inequality follows fro 8. So it follows that P S GP Ω = P Ω G P S 8 log1 + ε 0 p and P Ω G P Sl P Ω G P Sl P S + P Ω G P S 8 log L l Gy F σ in Gy + P Ω G P S p 1/ ε 0 8 log 8 log1 + ε log1 + ε 0 p 8 log1 + ε 0 p 1/, where the second inequality follows fro 7 and Le. 6, the third inequality follows fro 10. Finally, P Sl GG P Sl p 1 P Sl GP Ω G P Sl P S GG P S p 1 P S GP Ω G P S + P S P Sl GG P Sl + P S GG P S P Sl + p 1 P S P Sl GP Ω G P Sl + p 1 P S GP Ω G P S P Sl ε L l Gy σ in Gy 4ε 0, which copletes the proof of 1. + p 1 L l Gy σ in Gy Proof of Theore 3. First note that L l+1 = T r W l, where So we have P Ω G P Sl + P S GP Ω W l = P Sl Hx l + p 1 P Ω x x l = P Sl Gy l + p 1 P Ω y y l. L l+1 Gy F W l L l+1 F + W l Gy F W l Gy F = P Sl Gy l + p 1 P Ω y y l Gy F P Sl Gy Gy F + P Sl G p 1 P Sl GP Ω y l y F = I P Sl L l Gy F + P Sl GG p 1 P Sl GP Ω G L l Gy F I P Sl L l Gy F + P Sl GG P Sl p 1 P Sl GP Ω G P Sl L l Gy F + P Sl GG I P Sl L l Gy F + p 1 P Sl GP Ω G I P Sl L l Gy F, := I 1 + I + I 3 + I 4, 6

7 where the second inequality coes fro the fact that L l+1 is the best ran r approxiation to W l, the second equality follows fro I P Sl L l = 0, y l = G L l and G G = I. Let us first assue 10 holds. Then the application of Le. 6 gives I 1 + I 3 + I 4 4 Ll Gy F σ in Gy ε 0 L l Gy F, + p 1 P Ω G P Sl L l Gy F σ in Gy L l Gy F where the last inequality follows fro 8, 11 and the fact P Sl GP Ω = P Ω G P Sl. Moreover, 1 iplies I 8ε 0 L l Gy F. Therefore putting the bounds for I 1, I, I 3, and I 4 together gives L l+1 Gy F ν L l Gy F, where ν = 10ε 0 < 1. Since 10 holds for l = 0 by the assuption of Th. 3 and L l Gy F is a contractive sequence, 10 holds for all l 0. Thus y l y = G L l Gy L l Gy F ν l L 0 Gy F, where we have utilized the facts y l = G L l, G G = I and G 1.. Proofs of Lea and Theore 1 Proof of Lea. Recall that L 0 = T r p 1 HP Ω x = T r p 1 GP Ω y and Hx = Gy. Let us first bound p 1 GP Ω y Gy. Since p =, we have p 1 GP Ω y Gy = y a H a 1 Gy := Z a. Because each a is drawn uniforly fro {0,, 1}, it is trivial that E Z a = 0. Moreover, we have E Z a Za = E y a H a Ha 1 GyGy = 1 a=0 y a H a H a 1 GyGy = C 1 GyGy, where C is a diagonal atrix which corresponds to the diagonal part of GyGy. Therefore E Z a Za C Gy, 7

8 where Gy denotes the axiu row l nor of Gy. Siilarly we can get E Za Z a Gy. The definition of H a iplies H a 1 wa. So Z a y a H a a=0 y a H a D 1 y. By atrix Bernstein inequality in Le. 7, one can show that there exists a universal constant C > 0 such that log Z a C ax { Gy, Gy } + log D 1 y with probability at least 1. Consequently on the sae event we have L 0 Gy L 0 p 1 GP Ω y + p 1 GP Ω y Gy p 1 GP Ω y Gy log C ax { Gy, Gy } + log D 1 y. 13 Thus it only reains to bound ax { Gy, Gy } and D 1 y in ters of Gy. Fro Gy = Hx = UΣV = E L DER T, we get Gy = ax i = ax i e i Gy = ax e i UΣV ax e i U Σ i i U i,: Gy µ 0c s r Gy, 14 where the last inequality follows fro Le. 1. Siilarly we also have Gy µ 0c s r Gy. 15 The infinity nor of D 1 y can be bounded as follows D 1 y = Gy = ax e i Gye j ax e i E L D E T Re j i,j i,j r D r E L Gy ER T µ 0 c s r Gy, 16 where the last inequality follows fro the µ 0 -incoherence of Gy. Finally inserting 14, 15 and 16 into 13 gives provided µ 0 c s r log. µ0 c s r log L 0 Gy C Gy 8

9 Proof of Theore 1. Following fro 5, we only need to verify when the three conditions in Th. 3 are satisfied. Lea 4 iplies 7 holds with probability at least 1. Leas 1 and 5 guarantees 8 is true with probability at least 1 if Cε 0 µ 0c s r log for a sufficiently large nuerical constant C > 0. Siilarly 9 can be satisfied with probability at least 1 if C1 + ε 0 ε 1 0 µ1/ 0 c 1/ s κr 1/ log 3/ following Le. and the fact L 0 Gy F r L0 Gy, where κ denotes the condition nuber of Gy. Taing an upper bound on the nuber of easureents copletes the proof of Th Proofs of Lea 3 and Theore The proof of Le. 3 relies on the following estiation of PŜl G p 1 I G P Ωl+1, P U PÛl which is a generalization of the asyetric restricted isoetry property [6] fro atrix copletion to low ran Hanel atrix copletion. Lea 9. Assue there exists a nuerical constant µ such that and PÛl H a F µc sr, P U H a F µc sr, P Vl H a F µc sr, 17 P V H a F µc sr. 18 for all 0 a 1. Let Ω l+1 = {a = 1,, } be a set of indices sapled with replaceent. If is independent of U, V, P Ωl+1 Ûl and V l, then PŜl G I p 1 G 160µcs r log P P Ωl+1 U PÛl with probability at least 1 provided Proof. Since for any Z C p 1p p 3 q 1 q q 3 PŜl GP Ωl+1 G P U PÛl Z = we can rewrite PŜl GP Ωl+1 G P U PÛl as Define the rando operator PŜl GP Ωl+1 G P U PÛl = µc sr log. Z, P U PÛl H a P Sl H a, P Sl H a P U PÛl H a. R a = PŜl H a P U PÛl H a 1 P GG P Ŝl U PÛl. 9

10 Then it is easy to see that E R a = 0. By assuption, for any 0 a 1, So PŜl H a F PÛl H a F + P Vl H a F µc sr. R a PŜl H a P U PÛl H a + 1 F F PŜl GG P U PÛl 5µc s r. ext let us bound ER a R a as follows ER a R a = E P U PÛl H a P F Ŝ l H a PŜl H a 1 P Ŝ l GG P U PÛl GG PŜl E P U PÛl H a P F Ŝ l H a PŜl H a + 4 4µc sr 4 E PŜl H a PŜl H a + = 4µc sr PŜl GG 4 PŜl + 8µc sr. This iplies E R a R a ER a R a 8µc sr. We can siilarly obtain E R a R a 1µc sr. So the application of the atrix Bernstein inequality in Le. 7 gives { } t / P R a t p 1 p p 3 q 1 q q 3 exp If t Setting t = 4 5, then { P R a t } 160µc s r log p 1 p p 3 q 1 q q 3 exp gives { P R a t 10 } t / 0µc s r. 1µc s r + 5µcsr t/3 exp. t / 0µc s r.

11 4 The condition t 5 iplies µc sr log. The proof is coplete because R a = PŜl G p 1 I G P P Ωl+1 U PÛl. The following lea fro [6] will also be used in the proof of Le. 3. Lea 10. Let L l = Ũl Σ l Ṽ l and Gy = UΣV be two ran r atrices which satisfy L l Gy σ ingy F 10. Assue U i,: µ 0c sr and V j,: µ 0c sr. Then the atrix L l = Tri µ0 L l = Ûl Σ l V l returned by Alg. 4 satisfies L l Gy 8κ L { l Gy and ax Û i,:, V j,: } 100µ 0c s r F F 81, where κ denotes the condition nuber of Gy. Proof of Lea 3. Let us first assue that L l Gy σ ingy F 56κ. 19 Then the application of Le. 10 iplies that L l Gy 8κ L { l Gy and ax Û i,:, V j,: } 100µ 0c s r F F 81 by noting that U i,: µ 0c sr calculation gives and V j,: µ 0c sr PÛl H a = Ûl H a = 1 i,: F F Γ a Ûl i Γ a 0 following fro Le. 1. Moreover, direct 100µ 0c s r 81, 1 where Γ a is the set of row indices for non-zero entries in H a with cardinality Γ a = w a. Siilarly, P Vl H a F 100µ 0c s r 81. Recall that y = Dx and Gy = Hx. Define ŷ l = D x l. Then ŷ l = G Ll and PŜl H x l + p 1 P Ωl+1 x x l = P Sl G ŷ l + p 1 P Ωl+1 y ŷ l. 11

12 Consequently, L l+1 Gy F P Sl G ŷ l + p 1 y ŷ P Ωl+1 l Gy F PŜl PŜl Gy Gy + G p 1 PŜl GP Ωl+1 ŷ l y F F I = PŜl Gy + PŜl GG p 1 PŜl G F Ll Gy F GP Ωl+1 F I PŜl Ll Gy + PŜl GG PŜl p 1 PŜl G F PŜl Ll Gy GP Ωl+1 + PŜl G I p 1 G I PŜl L P Ωl+1 l Gy := I 5 + I 6 + I 7. F The first ite I 5 can be bounded as L l Gy I 5 σ in Gy F 1 L l Gy, F which follows fro Le. 6, the left inequality of 0 and the assuption 19. The application of Le. 5 together with 1 and iplies 300µ0 c s r log I 6 81 L 300µ0 c s r log l Gy 16κ F 81 L l Gy F with probability at least 1. To bound I 7, first note that I PŜl Ll Gy = I PŜl Gy = I ÛlÛ l Gy I V l V l = UU ÛlÛ l Ll Gy I V l V l = P U PÛl I P Ll Vl Gy. Therefore I 7 = PŜl G I p 1 G F I PŜl P P Ωl+1 U PÛl I P Ll Vl Gy PŜl G I p 1 G I PŜl P P Ωl+1 U PÛl Ll Gy F 16000µ0 c s r log 16κ 81 L l Gy F with probability at least 1, where the last inequality follows fro Le. 9 and the left inequality of 0. Putting the bounds for I 5, I 6 and I 7 together gives L 1 l+1 Gy F + 36κ µ0 c s r log Ll Gy 5 F 6 L l Gy F with probability at least 1 provided Cµ 0 c s κ r log for a sufficiently large universal constant C. Clearly on the sae event, 19 also holds for the l + 1-th iteration. 1

13 Since L 0 = T r p 1 HP Ω0 x, 19 is valid for l = 0 with probability at least 1 provides Cµ 0 c s κ 6 r log for soe nuerical constant C > 0. Taing the upper bound on the nuber of easureents copletes the proof of Le. 3 by noting Hx = Gy. Proof of Theore. The third condition 9 in Th. 3 can be satisfied with probability at least 1 L + 1 if we tae L = 6 log log 16ε 0. So the theore can be proved by cobining this result together with Les. 4 and 5. References [1] J. F. Cai, T. Wang, and K. Wei, Fast and provable algoriths for spectrally sparse signal reconstruction via low-ran hanel atrix copletion, arxiv preprint arxiv: , 016. [] Y. Chen and Y. Chi, Robust spectral copressed sensing via structured atrix copletion, Inforation Theory, IEEE Transactions on, , pp [3] W. Liao, MUSIC for ultidiensional spectral estiation: Stability and super-resolution, Signal Processing, IEEE Transactions on, , pp [4] B. Recht, A sipler approach to atrix copletion, The Journal of Machine Learning Research, 1 011, pp [5] J. A. Tropp, User-friendly tail bounds for sus of rando atrices, Foundations of coputational atheatics, 1 01, pp [6] K. Wei, J. F. Cai, T. F. Chan, and S. Leung, Guarantees of Rieannian optiization for low ran atrix copletion, arxiv preprint arxiv: ,