Robust Spectral Compressed Sensing via Structured Matrix Completion Yuxin Chen, Student Member, IEEE, and Yuejie Chi, Member, IEEE

Transcription

1 6576 IEEE TRANSACTIONS ON INORMATION THEORY, VOL 60, NO 0, OCTOBER 04 Robust Spectral Copressed Sensing via Structured Matrix Copletion Yuxin Chen, Student Meber, IEEE, and Yuejie Chi, Meber, IEEE Abstract This paper explores the proble of spectral copressed sensing, which ais to recover a spectrally sparse signal fro a sall rando subset of its n tie doain saples The signal of interest is assued to be a superposition of r ultidiensional coplex sinusoids, while the underlying frequencies can assue any continuous values in the noralized frequency doain Conventional copressed sensing paradigs suffer fro the basis isatch issue when iposing a discrete dictionary on the ourier representation To address this issue, we develop a novel algorith, called enhanced atrix copletion EMaC), based on structured atrix copletion that does not require prior knowledge of the odel order The algorith starts by arranging the data into a low-rank enhanced for exhibiting ultifold Hankel structure, and then attepts recovery via nuclear nor iniization Under ild incoherence conditions, EMaC allows perfect recovery as soon as the nuber of saples exceeds the order of r log 4 n, and is stable against bounded noise Even if a constant portion of saples are corrupted with arbitrary agnitude, EMaC still allows exact recovery, provided that the saple coplexity exceeds the order of r log 3 n Along the way, our results deonstrate the power of convex relaxation in copleting a low-rank ultifold Hankel or Toeplitz atrix fro inial observed entries The perforance of our algorith and its applicability to super resolution are further validated by nuerical experients Index Ters Spectral copressed sensing, atrix copletion, Hankel atrices, Toeplitz atrices, basis isatch, off-grid copressed sensing, incoherence, super-resolution I INTRODUCTION A Motivation and Contributions A large class of practical applications features high-diensional signals that can be odeled or approxiated by a superposition of spikes in the spectral resp tie) doain, and involves estiation of the signal fro its tie resp frequency) doain saples Exaples include acceleration of edical iaging [], target localization in radar and sonar systes [], inverse scattering in seisic iaging [3], fluorescence icroscopy [4], channel estiation in wireless counications [5], analog-to-digital conversion [6], etc Manuscript received April 8, 03; revised March 7, 04; accepted July 8, 04 Date of publication July 9, 04; date of current version Septeber, 04 Y Chi was supported by the Ohio State University, Colubus, OH, USA Y Chen is with the Departent of Electrical Engineering, Stanford University, Stanford, CA USA e-ail: yxchen@stanfordedu) Y Chi is with Departent of Electrical and Coputer Engineering, Departent of Bioedical Inforatics, Ohio State University, Colubus, OH 430 USA e-ail: chi97@osuedu) Counicated by V Saligraa, Associate Editor for Signal Processing Color versions of one or ore of the figures in this paper are available online at Digital Object Identifier 009/TIT The data acquisition devices, however, are often liited by hardware and physical constraints, precluding sapling with the desired resolution It is thus of paraount interest to reduce sensing coplexity while retaining recovery accuracy In this paper, we investigate the spectral copressed sensing proble, which ais to recover a spectrally sparse signal fro a sall nuber of randoly observed tie doain saples The signal of interest x t) with abient diension n is assued to be a weighted su of ulti-diensional coplex sinusoids at r distinct frequencies { f i [0, ] K : i r}, where the underlying frequencies can assue any continuous values on the unit interval Spectral copressed sensing is closely related to the proble of haronic retrieval, which seeks to extract the underlying frequencies of a signal fro a collection of its tie doain saples Conventional ethods for haronic retrieval include Prony s ethod [7], ESPRIT [8], the atrix pencil ethod [9], the Tufts and Kuaresan approach [0], the finite rate of innovation approach [], [], etc These ethods routinely exploit the shift invariance of the haronic structure, naely, a consecutive segent of tie doain saples lies in the sae subspace irrespective of the starting point of the segent However, one weakness of these techniques if that they require prior knowledge of the odel order, that is, the nuber of underlying frequency spikes of the signal or at least an estiate of it Besides, these techniques heavily rely on the knowledge of the noise spectra, and are often sensitive against noise and outliers [3] Another line of work is concerned with Copressed Sensing CS) [4], [5] over a discrete doain, which suggests that it is possible to recover a signal even when the nuber of saples is far below its abient diension, provided that the signal enjoys a sparse representation in the transfor doain In particular, tractable algoriths based on convex surrogates becoe popular due to their coputational efficiency and robustness against noise and outliers [6], [7] urtherore, they do not require prior inforation on the odel order Nevertheless, the success of CS relies on sparse representation or approxiation of the signal of interest in a finite discrete dictionary, while the true paraeters in any applications are actually specified in a continuous dictionary The basis isatch between the true frequencies and the discretized grid [8] results in loss of sparsity due to spectral leakage along the Dirichlet kernel, and hence degeneration in the perforance of conventional CS paradigs In this paper, we develop an algorith, called Enhanced Matrix Copletion EMaC), that siultaneously exploits the IEEE Personal use is peritted, but republication/redistribution requires IEEE perission See for ore inforation

2 CHEN AND CHI: ROBUST SPECTRAL COMPRESSED SENSING 6577 shift invariance property of haronic structures and the spectral sparsity of signals Inspired by the conventional atrix pencil for [9], EMaC starts by arranging the data saples into an enhanced atrix exhibiting K -fold Hankel structures, whose rank is bounded above by the spectral sparsity r This way we convert the spectral sparsity into the low-rank structure without iposing any pre-deterined grid EMaC then invokes a nuclear nor iniization progra to coplete the enhanced atrix fro partially observed saples When a sall constant proportion of the observed saples are corrupted with arbitrary agnitudes, EMaC solves a weighted nuclear nor iniization and l nor iniization to recover the signal as well as the sparse corruption coponent The perforance of EMaC depends on an incoherence condition that depends only on the frequency locations regardless of the aplitudes of their respective coefficients The incoherence easure is characterized by the reciprocal of the sallest singular value of soe Gra atrix, which is defined by sapling the Dirichlet kernel at the wrap-around differences of all frequency pairs The signal of interest is said to obey the incoherence condition if the Gra atrix is well conditioned, which arises over a broad class of spectrally sparse signals including but not restricted to signals with well-separated frequencies We deonstrate that, under this incoherence condition, EMaC enables exact recovery fro Or log 4 n) rando saples, and is stable against bounded noise Moreover, EMaC adits perfect signal recovery fro Or log 3 n) rando saples even when a constant proportion of the saples are corrupted with arbitrary agnitudes inally, nuerical experients validate our theoretical findings, and deonstrate the applicability of EMaC in super resolution Along the way, we provide theoretical guarantees for low-rank atrix copletion of Hankel atrices and Toeplitz atrices, which is of great iportance in control, natural language processing, and coputer vision To the best of our knowledge, our results provide the first theoretical guarantees for Hankel atrix copletion that are close to the inforation theoretic liit B Connection and Coparison to Prior Work The K -fold Hankel structure, which plays a central role in the EMaC algorith, roots fro the traditional spectral estiation technique naed Matrix Enhanceent Matrix Pencil MEMP) [9] for ulti-diensional haronic retrieval The conventional MEMP algorith assues fully observed equi-spaced tie doain saples for estiation, and require prior knowledge on the odel order Cadzow s denoising ethod [0] also exploits the low-rank structure of the atrix pencil for for denoising line spectru, but the ethod is non-convex and lacks perforance guarantees The standard notation f n) = O gn)) eans that there exists a constant c > 0 such that f n) cgn); f n) = gn)) indicates that there are nuerical constants c, c > 0suchthatc gn) f n) c gn) When the frequencies of the signal indeed fall on a grid, CS algoriths based on l iniization [4], [5] assert that it is possible to recover the spectrally sparse signal fro Or log n) rando tie doain saples These algoriths adit faithful recovery even when the saples are containated by bounded noise [6], [] or arbitrary sparse outliers [7] When the inevitable basis isatch issue [8] is present, several reedies of CS algoriths have been proposed to itigate the effect [], [3] under rando linear projection easureents, although theoretical guarantees are in general lacking More recently, Candès and ernandez-granda [4] proposed a total-variation nor iniization algorith to super-resolve a sparse signal fro frequency saples at the low end of the spectru This algorith allows accurate super-resolution when the point sources are sufficiently separated, and is stable against noise [5] Inspired by this approach, Tang et al [6] then developed an atoic nor iniization algorith for line spectral estiation fro Or log r log n) rando tie doain saples, which enables exact recovery when the frequencies are separated by at least 4/n with rando aplitude phases Siilar perforance guarantees are later established in [7] for ulti-diensional frequencies However, these results are established under a rando signal odel, ie the coplex signs of the frequency spikes are assued to be iid drawn fro a unifor distribution The robustness of the ethod against noise and outliers is not established either In contrast, our approach yields deterinistic conditions for ulti-diensional frequency odels that guarantee perfect recovery with noiseless saples and are provably robust against noise and sparse corruptions We will provide detailed coparison with the approach of Tang et al after we forally present our results Nuerical coparison will also be provided in Section V-C for the line spectru odel Our algorith is inspired by recent advances of Matrix Copletion MC) [8], [9], which ais at recovering a lowrank atrix fro partial entries It has been shown [30] [3] that exact recovery is possible via nuclear nor iniization, as soon as the nuber of observed entries exceeds the order of the inforation theoretic liit This line of algoriths is also robust against noise and outliers [33], [34], and allows exact recovery even in the presence of a constant portion of adversarially corrupted entries [35] [37], which have found nuerous applications in collaborative filtering [38], edical iaging [39], [40], etc Nevertheless, the theoretical guarantees of these algoriths do not apply to the ore structured observation odels associated with the proposed ulti-fold Hankel structure Consequently, direct application of existing MC results delivers pessiistic saple coplexity, which far exceeds the degrees of freedo underlying the signal Preliinary results of this work have been presented in [4], where an additional strong incoherence condition was introduced that bore a siilar role as the traditional strong incoherence paraeter in MC [30] but lacked physical interpretations This paper reoves this condition and further iproves the saple coplexity

3 6578 IEEE TRANSACTIONS ON INORMATION THEORY, VOL 60, NO 0, OCTOBER 04 C Organization The rest of the paper is organized as follows The signal and sapling odels are described in Section II By restricting our attention to two-diensional -D) frequency odels, we present the enhanced atrix for and the associated structured atrix copletion algoriths The extension to ulti-diensional frequency odels is discussed in Section III-C The ain theoretical guarantees are suarized in Section III, based on the incoherence condition introduced in Section III-A We then discuss the extension to lowrank Hankel and Toeplitz atrix copletion in Section IV Section V presents the nuerical validation of our algoriths The proofs of Theores and 3 are based on duality analysis followed by a golfing schee, which are supplied in Section VI and Section VII, respectively Section VIII concludes the paper with a short suary of our findings as well as a discussion of potential extensions and iproveents inally, the proofs of auxiliary leas supporting our results are deferred to the appendices II MODEL AND ALGORITHM Assue that the signal of interest xt) can be odeled as a weighted su of K -diensional coplex sinusoids at r distinct frequencies f i [0, ] K, i r, ie xt) = r d i e jπ t, f i, t Z K ) i= It is assued throughout that the frequencies f i s are noralized with respect to the Nyquist frequency of xt) and the tie doain easureents are sapled at integer values We denote by d i s the coplex aplitudes of the associated coefficients, and, represents the inner product or concreteness, our discussion is ainly devoted to a -D frequency odel when K = This subsues line spectral estiation as a special case, and indicates how to address ultidiensional odels The algoriths for higher diensional scenarios closely parallel the -D case, which will be briefly discussed in Section III-C A -D requency Model Consider a data atrix X =[X k,l ] 0k<n,0l<n of abient diension n := n n, which is obtained by sapling the signal ) on a unifor grid ro ) each entry X k,l can be expressed as X k,l = xk, l) = r d i yi k zl i, ) i= where for any i i r) wedefine y i := exp jπ f i ) and z i := exp jπ f i ) { for soe frequency pairs f i = f i, f i ) i r } We can then express X in a atrix for as follows X = YDZ, 3) where the above atrices are defined as y y y r Y :=, 4) y n y n y n r z z z r Z :=, 5) z n z n z n r and D := diag [d, d,, d r ] 6) The above for 3) is soeties referred to as the Vandeonde decoposition of X Suppose that there exists a location set of size such that the X k,l is observed if and only if k, l) It is assued that is sapled uniforly at rando Define P X) as the orthogonal projection of X onto the subspace of atrices that vanish outside We ai at recovering X fro P X) B Matrix Enhanceent One ight naturally attept recovery by applying the low-rank MC algoriths [8], arguing that when r is sall, perfect recovery of X is possible fro partial easureents since X is low rank if r in{n, n } Specifically, this corresponds to the following algorith: iniize M M C n n subject to P M) = P X), 7) where M denotes the nuclear nor or su of all singular values) of a atrix M = [M k,l ] This is a convex relaxation paradig with respect to rank iniization However, naive MC algoriths [3] require at least the order of r ax n, n ) log n n ) saples in order to allow perfect recovery, which far exceeds the degrees of freedo which is r)) in our proble What is worse, since the nuber r of spectral spikes can be as large as n n, X ight becoe fullrank once r > in n, n ) This otivates us to seek other fors that better capture the haronic structure In this paper, we adopt one effective enhanced for of X based on the following two-fold Hankel structure The enhanced atrix X e with respect to X is defined as a k n k + ) block Hankel atrix X 0 X X n k X X X n k + X e :=, 8) X k X k X n where k k n ) is called a pencil paraeter Each block is a k n k + ) Hankel atrix defined such that for every l 0 l<n ): X l,0 X l, X l,n k X l, X l, X l,n k + X l :=, 9) X l,k X l,k X l,n

4 CHEN AND CHI: ROBUST SPECTRAL COMPRESSED SENSING 6579 where k n is another pencil paraeter This enhanced for allows us to express each block as X l = Z L Y l d DZ R, 0) where Z L, Z R and Y d are defined respectively as z z z r Z L :=, z k z k z k r z z n k z Z R := z n k, z r z n k r and Y d := diag [y, y,, y r ] Substituting 0) into 8) yields the following: Z L Z L Y d [ ] X e = D Z R, Y d Z R,, Y n k d Z R, ) }{{} Z L Y k d n k +)n k +)E R }{{} k k E L where E L and E R span the colun and row space of X e, respectively This iediately iplies that X e is low-rank,ie rank X e ) r ) This for is inspired by the traditional atrix pencil approach proposed in [9] and [9] to estiate haronic frequencies if all entries of X are available Thus, one can extract all underlying frequencies of X using ethods proposed in [9], as long as X can be faithfully recovered C The EMaC Algorith in the Absence of Noise We then attept recovery through the following Enhanceent Matrix Copletion EMaC) algorith: EMaC) iniize M e M C n n subject to P M) = P X), 3) where M e denotes the enhanced for of M Inotherwords, EMaC iniizes the nuclear nor of the enhanced for over all atrices copatible with the saples This convex progra can be rewritten into a seidefinite progra SDP) [4] iniize M C n n Tr ) Q + Tr ) Q subject to P M) = P X), [ Q M ] e 0, M e Q Note that the lth 0 l < n )rowx l of X can be expressed as X l = [ y l,, ] yl r DZ [ = y l d,, yr l d ] r Z, and hence we only need to find the Vandeonde decoposition for X 0 and then replace d i by yi l d i which can be solved using off-the-shelf solvers in a tractable anner see [4]) It is worth entioning that EMaC has a siilar coputational coplexity as the atoic nor iniization ethod [6] when restricted to the -D frequency odel Careful readers will reark that the perforance of EMaC ust depend on the choices of the pencil paraeters k and k In fact, if we define a quantity c s := ax { n n k k, n n n k + )n k + ) } 4) that easures how close X e is to a square atrix, then it will be shown later that the required saple coplexity for faithful recovery is an increasing function of c s In fact, both our theory and epirical experients are in favor of a sall c s, corresponding to the choices k = n ), n k + = n ), k = n ),andn k + = n ) D The Noisy-EMaC Algorith With Bounded Noise In practice, easureents are often containated by a certain aount of noise To ake our odel and algorith ore practically applicable, we replace our easureents by X o =[Xk,l o ] 0k<n,0l<n through the following noisy odel Xk,l o = X k,l + N k,l, k, l), 5) where Xk,l o is the observed k, l)-th entry, and N = [N k,l ] 0k<n,0l<n denotes soe unknown noise We assue that the noise agnitude is bounded by a known aount P N) δ, where denotes the robenius nor In order to adapt our algorith to such noisy easureents, one wishes that sall perturbation in the easureents should result in sall variation in the estiate Our algorith is then odified as follows Noisy-EMaC) : iniize M e M C n n subject to P M X o ) δ 6) That said, the algorith searches for a candidate with iniu nuclear nor aong all signals close to the easureents E The Robust-EMaC Algorith With Sparse Outliers An outlier is a data saple that can deviate arbitrarily fro the true data point Practical data saples one collects ay contain a certain portion of outliers due to abnoral behavior of data acquisition devices such as aplifier saturation, sensor failures, and alicious attacks A desired recovery algorith should be able to autoatically prune all outliers even when they corrupt up to a constant portion of all data saples Specifically, suppose that our easureents X o are given by X o k,l = X k,l + S k,l, k, l), 7) where X o k,l is the observed k, l)-th entry, and S = [S k,l ] 0k<n,0l<n denotes the outliers, which is assued to be a sparse atrix supported on soe location set dirty The sapling odel is forally described as follows

5 6580 IEEE TRANSACTIONS ON INORMATION THEORY, VOL 60, NO 0, OCTOBER 04 ) Suppose that is obtained by sapling entries uniforly at rando, and define ρ := n n ) Conditioning on k, l), theevents{k, l) dirty } are independent with conditional probability P{k, l) dirty k, l) } =τ for soe sall constant corruption fraction 0 < τ< 3) Define clean := \ dirty as the location set of uncorrupted easureents EMaC is then odified as follows to accoodate sparse outliers: Robust-EMaC) iniize M e + λ Ŝ e M,Ŝ C n n ) subject to P M + Ŝ = P X + S), 8) where λ > 0 is a regularization paraeter that will be specified later As will be shown later, λ can be selected in a paraeter-free fashion We denote by M e and Ŝ e the enhanced for of M and Ŝ, respectively Here, Ŝ e := vecŝ e ) represents the eleentwise l -nor of Ŝ e Robust-EMaC prootes the low-rank structure of the enhanced data atrix as well as the sparsity of the outliers via convex relaxation with respective structures Notations Before continuing, we introduce a few notations that will be used throughout Let the singular value decoposition SVD) of X e be X e = U V Denote by { T := UM + MV : M C n k +)n k +) r, M C k k r } 9) the tangent space with respect to X e,andt the orthogonal copleent of T Denote by P U resp P V, P T ) the orthogonal projection onto the colun resp row, tangent) space of X e,ieforanym, P U M) = UU M, P V M) = MVV, and P T = P U + P V P U P V We let P T = I P T be the orthogonal copleent of P T, where I denotes the identity operator Denote by M, M and M the spectral nor operator nor), robenius nor, and nuclear nor of M, respectively Also, M and M are defined to be the eleentwise l and l nor of M Denote by e i the i th standard basis vector Additionally, we use sgn M) to denote the eleentwise coplex sign of M On the other hand, we denote by e k, l) the set of locations of the enhanced atrix X e containing copies of X k,l Due to the Hankel or ulti-fold Hankel structures, one can easily verify the following: each location set e k, l) contains at ost one index in any given row of the enhanced for, and at ost one index in any given colun or each k, l) ig a) The -D Dirichlet kernel when k = k = k = 6; b) The epirical distribution of the iniu eigenvalue σ in G L ) for various choices of k with respect to the sparsity level [n ] [n ],weusea k,l) to denote a basis atrix that extracts the average of all entries in e k, l) Specifically, { ) Ak,l) α,β := e k,l), if α, β) e k, l), 0) 0, else We will use throughout as a short-hand notation ω k,l := e k, l) ) III MAIN RESULTS This section delivers the following encouraging news: under ild incoherence conditions, EMaC enables faithful signal recovery fro a inial nuber of tie-doain saples, even when the saples are containated by bounded noise or a constant portion of arbitrary outliers A Incoherence Measure In general, atrix copletion fro a few entries is hopeless unless the underlying structure is sufficiently uncorrelated with the observation basis This inspires us to introduce certain incoherence easures To this end, we define the -D Dirichlet kernel as Dk, k, f ) := e jπk f ) e jπk f ) k k e jπ f e jπ f, ) where f = f, f ) [0, ] ig a) illustrates the aplitude of Dk, k, f ) when k = k = 6 The value of Dk, k, f ) decays inverse proportionally with respect to the frequency f SetG L and G R to be two r r Gra atrices such that their entries are specified respectively by G L ) i,l = Dk, k, f i f l ), G R ) i,l = Dn k +, n k +, f i f l ), where the difference f i f l is understood as the wrap-around distance in the interval [ /, /) Siple anipulation reveals that G L = E L E L, G R = E R E R), where E L and E R are defined in )

6 CHEN AND CHI: ROBUST SPECTRAL COMPRESSED SENSING 658 Our incoherence easure is then defined as follows Definition Incoherence): A atrix X is said to obey the incoherence property with paraeter μ if σ in G L ) μ and σ in G R ) μ 3) where σ in G L ) and σ in G R ) represent the least singular values of G L and G R, respectively The incoherence easure μ only depends on the locations of the frequency spikes, irrespective of the aplitudes of their respective coefficients The signal is said to satisfy the incoherence condition if μ scales as a sall constant, which occurs when G L and G R are both well-conditioned Our incoherence condition naturally requires certain separation aong all frequency pairs, as when two frequency spikes are closely located, μ gets undesirably large As shown in [43, Th ], a separation of about /n for line spectru is sufficient to guarantee the incoherence condition to hold However, it is worth ephasizing that such strict separation is not necessary as required in [6], and thereby our incoherence condition is applicable to a broader class of spectrally sparse signals To give the reader a flavor of the incoherence condition, we list two exaples below or ease of presentation, we assue below -D frequency odels with n = n Note, however, that the asyetric cases and general K -diensional frequency odels can be analyzed in the sae anner Rando frequency locations: suppose that the r frequencies are generated uniforly at rando, then the iniu pairwise ) separation can be crudely bounded by Ifn r log n r 5 log n, then a crude bound reveals that i = i, ) y k i y i ax k yi, ) z k i z i y i k zi z i r n r /4 holds with high probability, indicating that the offdiagonal entries of G L and G R are uch saller than /r in agnitude Siple anipulation then allows us to conclude that σ in G L ) and σ in G R ) are bounded below by positive constants ig b) shows the iniu eigenvalue of G L for different k = k = k = 6, 36, 7 when the spikes are randoly generated and the nuber of spikes is given as the sparsity level The iniu eigenvalue of G L gets closer to one as k grows, confiring our arguent Sall perturbation off the grid: suppose that all frequencies are within a distance at ost fro soe ) grid points l k, l k 0 l < k, 0 l < k ) One can verify that i = i, ) y k i y i ax k yi, ) z k i z i y i k zi z i < r, and hence the agnitude of all off-diagonal entries of G L and G R are no larger than /4r) This iediately suggests that σ in G L ) and σ in G R ) are lower bounded by 3/4 Note, however, that the class of incoherent signals are far beyond the ones discussed above B Theoretical Guarantees With the above incoherence easure, the ain theoretical guarantees are provided in the following three theores each accounting for a distinct data odel: ) noiseless easureents, ) easureents containated by bounded noise, and 3) easureents corrupted by a constant proportion of arbitrary outliers ) Exact Recovery ro Noiseless Measureents: Exact recovery is possible fro a inial nuber of noise-free saples, as asserted in the following theore Theore : Let X be a data atrix of for 3), and the rando location set of size Suppose that the incoherence property 3) holds and that all easureents are noiseless Then there exists a universal constant c > 0 such that X is the unique solution to EMaC with probability exceeding n n ), provided that > c μ c s r log 4 n n ) 4) Theore asserts that under soe ild deterinistic incoherence condition such that μ scales as a sall constant, EMaC adits prefect recovery as soon as the nuber of easureents exceeds Or log 4 n n )) Since there are r) degrees of freedo in total, the lower bound should be no saller than r) This deonstrates the orderwise optiality of EMaC except for a logarithic gap We note, however, that the polylog factor ight be further refined via finer tuning of concentration of easure inequalities It is worth ephasizing that while we assue rando observation odels, the data odel is assued deterinistic This differs significantly fro [6], which relies on randoness in both the observation odel and the data odel In particular, our theoretical perforance guarantees rely solely on the frequency locations irrespective of the associated aplitudes In contrast, the results in [6] require the phases of all frequency spikes to be iid drawn in a unifor anner in addition to a separation condition Reark : Theore significantly strengthens our prior results reported in [4] by iproving the required saple coplexity fro O μ c s r poly logn n ) ) to O μ c s rpoly logn n )) ) Stable Recovery in the Presence of Bounded Noise: Our ethod enables stable recovery even when the tie doain saples are noisy copies of the true data Here, we say the recovery is stable if the solution of Noisy-EMaC is close to the ground truth in proportion to the noise level To this end, we provide the following theore, which is a counterpart of Theore in the noisy setting, whose proof is inspired by [44] Theore : Suppose X o is a noisy copy of X that satisfies P X X o ) δ Under the conditions of Theore, the solution to Noisy-EMaC in 6) satisfies ˆX e X e 5n 3 n3 δ 5) with probability exceeding n n )

7 658 IEEE TRANSACTIONS ON INORMATION THEORY, VOL 60, NO 0, OCTOBER 04 Theore reveals that the recovered enhanced atrix which contains n n ) entries) is close to the true enhanced atrix at high SNR In particular, the average entry inaccuracy of the enhanced atrix is bounded above by On 3 n3 δ), aplified by the subsapling factor In practice, one is interested in an estiate of X, which can be obtained naively by randoly selecting an entry in e k, l) as ˆX k,l,thenwehave ˆX X ˆX e X e This yields that the per-entry noise of ˆX is about On 5n5 δ), which is further aplified due to enhanceent by a polynoial factor However, this factor arises fro an analysis artifact due to our siple strategy to deduce ˆX fro Xˆ e, and ay be elevated We note that in nuerical experients, Noisy-EMaC usually generates uch better estiates, usually by a polynoial factor The practical applicability will be illustrated in Section V It is worth entioning that to the best of our knowledge, our result is the first stability result with partially observed data for spectral copressed sensing off the grid While the atoic nor approach is near-iniax with full data [45], it is not clear how it perfors with partially observed data 3) Robust Recovery in the Presence of Sparse Outliers: Interestingly, Robust-EMaC can provably tolerate a constant portion of arbitrary outliers The theoretical perforance is forally suarized in the following theore Theore 3: Let X be a data atrix with atrix for 3), and a rando location set of size Set λ =, logn n ) and assue τ 0 is soe sall positive constant Then there exist a nuerical constant c > 0 depending only on τ such that if 3) holds and > c μ c s r log 3 n n ), 6) then Robust-EMaC is exact, ie the iniizer ˆM, Ŝ) satisfies ˆM = X, with probability exceeding n n ) Reark : Note that τ 0 is not a critical threshold In fact, one can prove the sae theore for a larger τ eg τ 05) with a larger absolute constant c However, to allow even larger τ eg in the regie where τ 50%), we need the sparse coponents exhibit rando sign patterns Theore 3 specifies a candidate choice of the regularization paraeter λ that allows recovery fro a few saples, which only depends on the size of but is otherwise paraeter-free In practice, however, λ ay better be selected via cross validation urtherore, Theore 3 deonstrates the possibility of robust recovery under a constant proportion of sparse corruptions Under the sae ild incoherence condition as for Theore, robust recovery is possible fro O r log 3 n n ) ) saples, even when a constant proportion of the saples are arbitrarily corrupted As far as we know, this provides the first theoretical guarantees for separating sparse easureent corruptions in the off-grid copressed sensing setting C Extension to Higher-Diensional and Daping requency Models By letting n = the above -D frequency odel reverts to the line spectru odel The EMaC algorith and the ain results iediately extend to higher diensional frequency odels without difficulty In fact, for K -diensional frequency odels, one can arrange the original data into a K -fold Hankel atrix of rank at ost r or instance, consider a 3-D odel such that X l,l,l 3 = r i= d i y l i z l i w l 3 i, l, l, l 3 ) [n ] [n ] [n 3 ] An enhanced for can be defined as a 3-fold Hankel atrix such that X 0,e X,e X n3 k 3,e X,e X,e X n3 k 3 +,e X e :=, X k3,e X k,e X n3,e where X i,e denotes the -D enhanced for of the atrix consisting of all entries X l,l,l 3 obeying l 3 = i One can verify that X e is of rank at ost r, and can thereby apply EMaC on the 3-D enhanced for To suarize, for K -diensional frequency odels, EMaC resp Noisy-EMaC, Robust-EMaC) searches over all K -fold Hankel atrices that are consistent with the easureents The theoretical perforance guarantees can be siilarly extended by defining the respective Dirichlet kernel in 3-D and the coherence easure In fact, all our analyses can be extended to handle daping odes, when the frequencies are not of tie-invariant aplitudes We oit the details for conciseness IV STRUCTURED MATRIX COMPLETION One proble closely related to our ethod is copletion of ulti-fold Hankel atrices fro a sall nuber of entries While each spectrally sparse signal can be apped to a low-rank ulti-fold Hankel atrix, it is not clear whether all ulti-fold Hankel atrices of rank r can be written as the enhanced for of a signal with spectral sparsity r Therefore, one can think of recovery of ulti-fold Hankel atrices as a ore general proble than the spectral copressed sensing proble Indeed, Hankel atrix copletion has found nuerous applications in syste identification [46], [47], natural language processing [48], coputer vision [49], agnetic resonance iaging [50], etc There has been several work concerning algoriths and nuerical experients for Hankel atrix copletions [46], [47], [5] However, to the best of our knowledge, there has been little theoretical guarantee that addresses directly Hankel atrix copletion Our analysis fraework can be straightforwardlyadapted to the general K -fold Hankel atrix copletions Below we present the perforance guarantee for the two-fold Hankel atrix copletion without loss of generality Notice that we need to odify the definition of μ as stated in the following theore Theore 4: Consider a two-fold Hankel atrix X e of rank r The bounds in Theores, and 3 continue to hold, if the incoherence μ is defined as the sallest nuber that satisfies { ax U A k,l), A k,l)v μ c s r } 7) k,l) [n ] [n ] n n

8 CHEN AND CHI: ROBUST SPECTRAL COMPRESSED SENSING 6583 ig Phase transition plots where frequency locations are randoly generated The plot a) concerns the case where n = n =, whereas the plot b) corresponds to the situation where n = n = 5 The epirical success rate is calculated by averaging over 00 Monte Carlo trials Condition 7) requires that the left and right singular vectors are sufficiently uncorrelated with the observation basis In fact, condition 7) is a weaker assuption than 3) It is worth entioning that a low-rank Hankel atrix can often be converted to its low-rank Toeplitz counterpart, by reversely ordering all rows of the Hankel atrix Both Hankel and Toeplitz atrices are effective fors that capture the underlying haronic structures Our results and analysis fraework extend to low-rank Toeplitz atrix copletion proble without difficulty ig 3 The reconstruction NMSE with respect to δ for a dataset with n = n =, r = 4and = 50 revealed to the algorith, while the vertical axis corresponds to the spectral sparsity level r The epirical success rate is reflected by the color of each cell It can be seen fro the plot that the nuber of saples grows approxiately linearly with respect to the spectral sparsity r, and that the slopes of the phase transition lines for two cases are approxiately the sae These observations are in line with our theoretical guarantee in Theore This phase transition diagras justify the practical applicability of our algorith in the noiseless setting V NUMERICAL EXPERIMENTS In this section, we present nuerical exaples to evaluate the perforance of EMaC and its variants under different scenarios We further exaine the application of EMaC in iage super resolution inally, we propose an extension of singular value thresholding SVT) developed by Cai et al [5] that exploits the ulti-fold Hankel structure to handle larger scale data sets A Phase Transition in the Noiseless Setting To evaluate the practical ability of the EMaC algorith, we conducted a series of nuerical experients to exaine the phase transition for exact recovery Let n = n, and we take k = k = n + )/ which corresponds to the sallest c s or each r, ) pair, 00 Monte Carlo trials were conducted We generated a spectrally sparse data atrix X by randoly generating r frequency spikes in [0, ] [0, ], and sapled a subset of size entries uniforly at rando The EMaC algorith was conducted using the convex prograing odeling software CVX with the interior-point solver SDPT3 [53] Each trial is declared successful if the noralized ean squared error NMSE) satisfies ˆX X / X 0 3, where ˆX denotes the estiate returned by EMaC The epirical success rate is calculated by averaging over 00 Monte Carlo trials ig illustrates the results of these Monte Carlo experients when the diensions 3 of X are and 5 5 The horizontal axis corresponds to the nuber of saples 3 We choose the diension of X to be odd siply to yield a squared atrix X e In fact, our results do not rely on n or n being either odd or prie We note that when n and n are known to be prie nubers, there ight exist coputationally cheaper ethods to enable perfect recovery see [54]) B Stable Recovery ro Noisy Data ig 3 further exaines the stability of the proposed algorith by perforing Noisy-EMaC with respect to different paraeter δ on a noise-free dataset of r = 4 coplex sinusoids with n = n = The nuber of rando saples is = 50 The reconstructed NMSE grows approxiately linear with respect to δ, validating the stability of the proposed algorith C Coparison With Existing Approaches for Line Spectru Estiation Suppose that we randoly observe 64 entries of an n-diensional vector n = 7) coposed of r = 4 odes or such -D signals, we copare EMaC with the atoic nor approach [6] as well as basis pursuit [55] assuing a grid of size or the atoic nor and the EMaC algorith, the odes are recovered via linear prediction using the recovered data [56] ig 4 deonstrates the recovery of ode locations for three cases, naely when a) all the odes are on the DT grid along the unit circle; b) all the odes are on the unit circle except two closely located odes that are off the presued grid; c) all the odes are on the unit circle except that one of the two closely located odes is a daping ode with aplitude 099 In all cases, the EMaC algorith successfully recovers the underlying odes, while the atoic nor approach fails to recover daping odes, and basis pursuit fails with both off-the-grid odes and daping odes We further copare the phase transition of the EMaC algorith and the atoic nor approach in [6] for line spectru estiation We assue a -D signal of length n = n = 7 and the pencil paraeter k of EMaC is chosen to be 64 The phase transition experients are conducted in the sae

9 6584 IEEE TRANSACTIONS ON INORMATION THEORY, VOL 60, NO 0, OCTOBER 04 ig 4 Recovery of ode locations when a) all the odes are on the DT grid along the unit circle; b) all the odes are on the unit circle except two closely located odes that are off the DT grid; c) all the odes are on the unit circle except that one of the two closely located odes is a daping ode The panels fro the upper left, clockwise, are the ground truth, the EMaC algorith, the atoic nor approach [6], and basis pursuit [55] assuing a grid of size ig 5 Phase transition for line spectru estiation of EMaC and the atoic nor approach [6] a) EMaC without iposing separation; b) atoic nor approach without iposing separation; c) atoic nor approach with separation anner as ig In the first case, the spikes are generated randoly as ig on a unit circle; in the second case, the spikes are generated until a separation condition is satisfied := in i =i f i f i 5/n ig 5 a) and b) illustrate the phase transition of EMaC and the atoic nor approach when the frequencies are randoly generated without iposing the separation condition The perforance of the atoic nor approach degenerates severely when the separation condition is not et; on the other hand, the EMaC gives a sharp phase transition siilar to the D case When the separation condition is iposed, the phase transition of the atoic nor approach greatly iproves as shown in ig 5 c), while the phase transition of EMaC still gives siilar perforance as in ig 5 a) We oit the actual phase transition in this case) However, it is worth entioning that when the sparsity level is relatively high, the required separation condition is in general difficult to be satisfied in practice In coparison, EMaC is less sensitive to the separation requireent D Robust Line Spectru Estiation Consider the proble of line spectru estiation, where the tie doain easureents are containated by a constant portion of outliers We conducted a series of Monte Carlo trials to illustrate the phase transition for perfect recovery of the ground truth The true data X is assued to be a 5-diensional vector, where the locations of the underlying frequencies are randoly generated The siulations were carried out again using CVX with SDPT3 ig 6 Robust line spectru estiation where ode locations are randoly generated: a) Phase transition plots when n = 5, and 0% of the entries are corrupted; the epirical success rate is calculated by averaging over 00 Monte Carlo trials b) Phase transition plots when n = 5, and all the entries are observed; the epirical success rate is calculated by averaging over 0 Monte Carlo trials ig 6a) illustrates the phase transition for robust line spectru estiation when 0% of the entries are corrupted, which showcases the tradeoff between the nuber of easureents and the recoverable spectral sparsity level r One can see fro the plot that is approxiately linear in r on the phase transition curve even when 0% of the easureents are corrupted, which validates our finding in Theore 3 ig 6b) illustrates the success rate of exact recovery when we obtain saples for all entry locations This plot illustrates the tradeoff between the spectral sparsity level and the nuber of outliers when all entries of the corrupted X o are observed It can be seen that there is a large region where exact recovery

10 CHEN AND CHI: ROBUST SPECTRAL COMPRESSED SENSING 6585 ig 7 A synthetic super resolution exaple, where the observation b) is taken fro the low-frequency coponents of the ground truth in a), and the reconstruction c) is done via inverse ourier transfor of the extrapolated high-frequency coponents can be guaranteed, deonstrating the power of our algoriths in the presence of sparse outliers E Synthetic Super Resolution The proposed EMaC algorith works beyond the rando observation odel in Theore ig 7 considers a synthetic super resolution exaple otivated by [4], where the ground truth in ig 7a) contains 6 point sources with constant aplitude The low-resolution observation in ig 7b) is obtained by easuring low-frequency coponents [ f lo, f lo ] of the ground truth Due to the large width of the associated point-spread function, both the locations and aplitudes of the point sources are distorted in the low-resolution iage We apply EMaC to extrapolate high-frequency coponents up to [ f hi, f hi ],where f hi /f lo = The reconstruction in ig 7c) is obtained via applying directly inverse ourier transfor of the spectru to avoid paraeter estiation such as the nuber of odes The resolution is greatly enhanced fro ig 7b), suggesting that EMaC is a proising approach for super resolution tasks The theoretical perforance is left for future work Singular Value Thresholding for EMaC The above Monte Carlo experients were conducted using the advanced SDP solver SDPT3 This solver and any other popular ones eg SeDuMi) are based on interior point ethods, which are typically inapplicable to large-scale data In fact, SDPT3 fails to handle an n n data atrix when n exceeds 9, which corresponds to a enhanced atrix One alternative for large-scale data is the first-order algoriths tailored to atrix copletion probles, eg the singular value thresholding SVT) algorith [5] We propose a odified SVT algorith in Algorith to exploit the Hankel structure In particular, two operators are defined as follows: D τt ) in Algorith denotes the singular value shrinkage operator Specifically, if the SVD of X is given by X = U V with = diag {σ i }), then D τt X) := Udiag { }) σ i τ t ) + V, where τ t > 0 is the soft-thresholding level Algorith Singular Value Thresholding for EMaC Input: The observed data atrix X o on the location set initialize: letx o e denote the enhanced for of P X o ); set M 0 = X 0 e and t = 0 repeat ) Q t D τt M t )) ) M t H X 0 Qt 3) t t + until convergence output ˆX as the data atrix with enhanced for M t In the K -diensional frequency odel, H X o Q t ) denotes the projection of Q t onto the subspace of enhanced atrices ie K -fold Hankel atrices) that are consistent with the observed entries Consequently, at each iteration, a pair ) Q t, M t is produced by first perforing singular value shrinkage and then projecting the outcoe onto the space of K -fold Hankel atrices that are consistent with observed entries The key paraeter that one needs to tune is the threshold τ t Unfortunately, there is no universal consensus regarding how to tweak the threshold for SVT type of algoriths One suggested choice is τ t = 0σ ax M t ) / t 0,whichworks well based on our epirical experients ig 8 illustrates the perforance of Algorith We generated a true 0 0 data atrix X through a superposition of 30 rando coplex sinusoids, and revealed 58% of the total entries ie = 600) uniforly at rando The noise was iid Gaussian giving a signal-to-noise aplitude ratio of 0 The reconstructed vectorized signal is superiposed on the ground truth in ig 8 The noralized reconstruction error was ˆX X / X = 0098, validating the stability of our algorith in the presence of noise VI PROO O THEOREMS AND 4 EMaC has siilar spirit as the well-known atrix copletion algoriths [8], [3], except that we ipose Hankel and ulti-fold Hankel structures on the atrices While [3] has presented a general sufficient condition for exact recovery see [3, Th 3]), the basis in our case does not exhibit

11 6586 IEEE TRANSACTIONS ON INORMATION THEORY, VOL 60, NO 0, OCTOBER 04 atrix copletion proble: ig 8 The perforance of SVT for Noisy-EMaC for a 0 0 data atrix that contains 30 rando frequency spikes 58% of all entries = 600) are observed with signal-to-noise aplitude ratio 0 Here, τ t = 0σ ax M t )/ t 0 epirically or concreteness, the reconstructed data against the true data for the first 00 tie instances after vectorization) are plotted desired coherence properties as required in [3], and hence these results cannot deliver inforative estiates when applied to our proble Nevertheless, the beautiful golfing schee introduced in [3] lays the foundation of our analysis in the sequel We also note that the analyses adopted in [8] and [3] rely on a desired joint incoherence property on UV,which has been shown to be unnecessary [3] or concreteness, the analyses in this paper focus on recovering haronically sparse signals as stated in Theore, since proving Theore is slightly ore involved than proving Theore 4 We note, however, that our analysis already entails all reasoning required for establishing Theore 4 A Dual Certification Denote by A k,l) M) the projection of M onto the subspace spanned by A k,l), and define the projection operator onto the space spanned by all A k,l) and its orthogonal copleent as A := A k,l), and A = I A 8) k,l) [n ] [n ] There are two coon ways to describe the randoness of : one corresponds to sapling without replaceent, and another concerns sapling with replaceent ie contains indices {a i [n ] [n ]:i } that are iid generated) As discussed in [3, Sec IIA], while both situations result in the sae order-wide bounds, the latter situation adits sipler analysis due to independence Therefore, we will assue that is a ulti-set possibly with repeated eleents) and a i s are independently and uniforly distributed throughout the proofs of this paper, and define the associated operators as A := A ai 9) i= We also define another projection operator A siilar to 9), but with the su extending only over distinct saples Its copleent operator is defined as A := A A Note that A M) = 0 is equivalent to A M) = 0 With these definitions, EMaC can be rewritten as the following general iniize M M subject to A M) = A X e), A M) = A X e ) = 0 30) To prove exact recovery of convex optiization, it suffices to produce an appropriate dual certificate, as stated in the following lea Lea : Consider a ulti-set that contains rando indices Suppose that the sapling operator A obeys P T AP T n n P T A P T 3) If there exists a atrix W satisfying A W) = 0, 3) P T W UV ) n, 33) n and PT W), 34) then X e is the unique solution to 30) or, equivalently, X is the unique iniizer of EMaC Proof: See Appendix B Condition 3) will be analyzed in Section VI-B, while a dual certificate W will be constructed in Section VI-C The validity of W as a dual certificate will be established in Sections VI-C VI-E These are the focus of the reaining section B Deviation of P T AP T n n P T A P T Lea requires that A be sufficiently incoherent with respect to the tangent space T The following lea quantifies the projection of each A k,l) onto the subspace T Lea : Under the hypothesis 3), one has UU A k,l) μ c s r n n, Ak,l) VV μ c s r n n, 35) for all k, l) [n ] [n ] or any a, b [n ] [n ], one has ωb 3μ c s r A b, P T A a 36) ω a n n Proof: See Appendix C Recognizing that 35) is the sae as 7), the following proof also establishes Theore 4 Note that Lea iediately leads to ) P T Ak,l) ) P U Ak,l) + ) P V Ak,l) μ c s r 37) n n As long as 37) holds, the fluctuation of P T A P T can be controlled reasonably well, as stated in the following lea This justifies Condition 3) as required by Lea Lea 3: Suppose that 37) holds Then for any sall constant 0 <ɛ, one has n n P T A P T P T AP T ɛ 38)