An Information-Theoretic Study for Noisy Multiple Measurement Vectors with Different Sensing Matrices

Transcription

1 > Aepted for publaton n IEEE ranaton on Informaton heory An Informaton-heoret tudy for oy Multple Meaurement Vetor wth Dfferent enng Matre angjun Park, am Yul Yu and Heung-o Lee*, enor Member, IEEE Abtrat In th paper, we tudy a upport et reontruton problem for multple meaurement vetor (MMV wth dfferent enng matre, where the gnal of nteret are aumed to be jontly pare and eah gnal ampled by t own enng matrx n the preene of noe. Ung mathematal tool, we develop upper and lower bound of the falure probablty of the upport et reontruton n term of the party, the ambent dmenon, the mum gnal-to-noe rato, the number of meaurement vetor, and the number of meaurement. hee bound an be ued to provde gudelne for deterng the ytem parameter for varou ompreed enng applaton wth noy MMV wth dfferent enng matre. Baed on the bound, we develop neeary and uffent ondton for a relable upport et reontruton. We nterpret thee ondton to provde theoretal explanaton regardng the beneft of takng more meaurement vetor. We then ompare our uffent ondton wth the extng reult for noy MMV wth the ame enng matrx. A a reult, we how that noy MMV wth dfferent enng matre may requre fewer meaurement for a relable upport et reontruton, under a ublnear party regme n a low noe-level enaro. Index erm ompreed enng, upport et reontruton, jont party truture, multple meaurement vetor model C I. IRODUCIO onventonally, gnal ened from enor uh a mrophone and magng deve are ampled followng the hannon and yqut amplng theory [] at a rate hgher than twe the maxmum frequeny for gnal reontruton. A the number of ample deded by th theory often large, the ample go through a ompreon tage before beng tored. herefore, takng numerou ample, where mot of them wll be darded n th tage, neffent. Beaue ompreed enng (C [] [7] remove the neffeny, C ha been appled n varou area uh a wrele ommunaton [8] [], petrometer [], multple nput multple output (MIMO radar [3], magnet reonane magng [4], and magng/gnal proeng [5] [7]. he C theory tate that gnal that are parely repreentable n a ertan ba are omprevely ampled and reontruted from what we thought nomplete nformaton. Let { } x be a -pare vetor wth a upport et x : = 0 whoe nde ndate the poton of the nonzero oeffent of x. It omprevely ampled by a model alled ngle meaurement vetor (MV a follow: y = Fx + n ( M M where y y a (noy meaurement vetor, F M a enng matrx, and n a noe vetor, whoe element are ndependent and dentally dtrbuted (..d Gauan wth a zero mean and a σ varane. One the upport et orretly reontruted, then ( an be well-poed, whh allow u to obtan an aurate etmate of x ung the leat quare approah. hu, we onder a upport et reontruton problem. A. Informaton-heoret Work for C wth MV Work [8] [3] have tuded the upport et reontruton problem from an nformaton-theoret perpetve. For relable upport et reontruton, uffent and neeary ondton were etablhed n the lnear and ublnear party regme. For a upport et reontruton, Wanwrght [8] ued the unon bound to etablh a uffent ondton on the number of meaurement M for a maxmum lkelhood (ML deoder and ued Fano nequalty [4] to obtan a neeary ondton on M. h ML deoder wa analyzed by Flether et al. [9] to etablh a neeary ondton on M. Aeron et al. [0] ued Fano nequalty to form neeary ondton on both M and σ. hen, they ued the unon bound to obtan uffent ondton on both M and σ for ther ub-optmal deoder. Akakaya and arokh [] ued the unon and the large devaton bound baed on empral entrope to get uffent ondton on M for ther jont typal deoder. hey ued the onvere of the hannel odng Manurpt reeved Aprl 0, 06; reved ovember 4, 06. h work wa upported by the atonal Reearh Foundaton of orea (RF grant funded by the outh orean government (RF-05RAAA h paper wa preented n part at the Internatonal ympoum on Informaton heory (II, Boton, UA, 0. angjun Park, am Yul Yu, and Heung-o Lee are wth the hool of Eletral Engneerng and Computer ene, Gwangju Inttute of ene and ehnology (GI, Gwangju, 6005 orea, e-mal: {jpark, nyyu, heungno}@gt.a.kr. he aterk * ndate the orrepondng author.

2 > Aepted for publaton n IEEE ranaton on Informaton heory theorem to get neeary ondton on M. arlett et al. [] extended th deoder [] wth the aumpton that the dtrbuton of the upport et provded. For a unform dtrbuton ae, ther neeary and uffent ondton are equvalent to thoe of []. However, they are better for a non-unform dtrbuton ae. arlett and Cevher [3] lked the upport et reontruton wth the problem of odng over a mxed hannel, where nformaton petrum method were ued to obtan neeary and uffent ondton on M. B. Informaton-heoret Work for C wth MMV C ha many applaton n wrele enor network (W [8] [] and MIMO radar [3]. In thee applaton, the gnal of nteret x, =,,, are often modeled a jontly -pare vetor, mplyng that = = =, where the upport et of x and =, whh referred to a a jont party truture. here are two model for amplng jontly -pare vetor. he frt model alled multple meaurement vetor (MMV wth the ame enng matrx [5], n whh they are ampled by the ame enng matrx. he eond model named a MMV wth dfferent enng matre [8][9], n whh eah one ampled by t own enng matrx. he author of [6] [8] have onduted nformaton-theoret reearhe to obtan ondton under whh the upport et of both the model wa reontruted wth a hgh probablty. In noy MMV wth the ame enng matrx, ang and ehora [6] ued the hypothe theory to obtan neeary and uffent ondton on both the number of meaurement M and the number of meaurement vetor, and proved that the ue probablty of the upport et reontruton nreae wth, f M = Ω log. Jn and Rao [7] exploted the ommunaton theory to etablh neeary and uffent ondton on M and demontrated the beneft of the jont party truture baed on ther ondton. A detaled omparon between the reult of our paper and [7] wll be preented n eton IV. Fnally, Duarte et al. [8] tuded noele MMV wth dfferent enng matre, and formed neeary and uffent ondton on M. However, t dffult to apply the ondton to noy MMV wth dfferent enng matre. Meanwhle, work [8][9][30] have preented ondton of pratal algorthm for a relable upport et reontruton. In noele MMV wth the ame enng matrx, Blanhard and Dave [30] obtaned ondton for a relable reontruton from rank aware orthogonal mathng purut (OMP. In noy MMV wth the ame enng matrx, m et al. [9] reated ompreve MUIC, and preented t uffent ondton. In noele MMV wth dfferent enng matre, Baron et al. [8] produed trval purut (P and dtrbuted ompreed enng-multaneou OMP (DC-OMP. By analyzng P wth the aumpton that eah enng matrx ontan..d. Gauan element and that the nonzero value of eah pare vetor are..d. Gauan varable, they demontrated that wth M meaurement, P reontrut the upport et a uffently large. hey onjetured that M + meaurement uffe for DC-OMP to reontrut the upport et a uffently large, baed on t empral reult. o the bet of our knowledge, no nformaton-theoret tudy ha been publhed to get neeary and uffent ondton for a relable upport et reontruton n noy MMV wth dfferent enng matre. Bede, thee ondton have not been provded from the pratal reovery algorthm for C wth noy MMV wth dfferent enng matre. C. Motvaton of th Paper C wth noy MMV wth dfferent enng matre ha been appled n many applaton and the beneft faltated by the jont party truture have been emprally reported n [0] [4]. In W, Caone et al. [0] ued the jont party truture to redue the number of tranmtted bt per enor and reported that eah enor an redue t tranmon ot. In magnet reonane magng (MRI, Wu et al. [4] modeled multple dffuon tenor mage (DI a jontly pare vetor. hey exploted the jont party truture to redue the number of ample per DI, whle retanng the reontruton qualty. Ung the jont party truture, they alo emprally reported that the reontruton qualty of eah DI an be mproved for a fxed number of ample per DI. o theoretally explan the above empral beneft faltated by the jont party truture, theoretal tool are requred to meaure the performane of C wth noy MMV wth dfferent enng matre. uh tool an be ued a gudelne for deterng the ytem parameter n varou C applaton wth noy MMV wth dfferent enng matre. For example, f the number of ample per DI fxed n the MRI [4], the theoretal tool may enable u to detere the number of DI requred for ahevng a gven reontruton qualty. hu, the frt motvaton of th paper to provde the theoretal tool by etablhng uffent and neeary ondton for a relable upport et reontruton. M ext, for noele MMV wth the ame enng matrx, let Y A = F x x x. Alo, for noele MMV wth M dfferent enng matre, let Y B = Fx Fx F x. hen, all the element of Y B are unorrelated beaue all the enng matre are ndependent. In ontrat, thoe of Y are orrelated beaue they are taken from the ame enng matrx. ow, we onder a ae where we et > and M >. hen, t lear that rank ( B = ( M, rank ( Y. herefore, for th ae, we onlude that rank > rank A A B A Y wth a hgh probablty and Y Y. h mple that a more relable upport et

3 > Aepted for publaton n IEEE ranaton on Informaton heory 3 reontruton an be expeted n noele MMV wth dfferent enng matre for th ae. hu, the eond motvaton to verfy th perepton n the preene of noe, by omparng our reult wth the extng one n noy MMV wth the ame enng matrx [7]. D. Contrbuton of th Paper he ontrbuton of th paper are a follow: Frt, we derve upper and lower bound of a falure probablty of the upport et reontruton from Lemma and, by explotng Fano nequalty [4] and the Chernoff bound [3]. We beleve that thee bound are ued for meaurng the performane of C wth noy MMV wth dfferent enng matre. ext, we develop neeary and uffent ondton for a relable upport et reontruton. heorem tate that M > + f ( R uffe to aheve a relable upport et reontruton n the lnear party regme,.e., β ( R M > + log f lm = 0,, and t alo tate that uffe to aheve a relable upport et reontruton n the ublnear party regme,.e., lm 0 =, where f an R nreang funton wth repet to the mum gnal-to-noe rato R defned n (4. ext, for a fnte,,, and R, heorem 3 tate that log log M < log ( + R neeary for a relable upport et reontruton. he neeary and uffent ondton an be ued a gudelne to detere the ytem parameter of C applaton wth noy MMV wth dfferent enng matre. Corollare and ndate that a relable upport et reontruton poble by takng uffently large meaurement vetor for a fxed M wth a low R value. For a fxed and, heorem how that M + meaurement uffe for reontrutng the upport et, a uffently large. hen, for a fxed,, and M = +, Corollary 3 provde a uffent ondton on for a relable upport et reontruton. We then provde theoretal explanaton of the beneft of the jont party truture, whh onform wth the empral reult of C applaton wth noy MMV wth dfferent enng matre [0][4]. Fnally, we ompare the uffent ondton ( wth the known one (6 for noy MMV wth the ame enng matrx [7]. herefore, we demontrate that f, noy MMV wth dfferent enng matre may requre fewer meaurement M for a relable upport et reontruton than noy MMV wth the ame enng matrx under a low noe-level enaro. It onfrm the uperorty of MMV wth dfferent enng matre. II. OAIO, YEM MODEL & PROBLEM FORMULAIO A. otaton he followng notaton wll be ued n the whole paper.., and denote the probablty, expetaton and (ovarane, repetvely.. A mall (aptal bold letter f (F a vetor (matrx. 3. A ub-vetor (ub-matrx formed by the element (olumn of f (F ndexed by a et denoted by f ( F. 4. For a gven matrx F, t nveron, tranpoe, trae and the th egenvalue are denoted by repetvely. Alo, t orthogonal projeton matrx defned by : = F, F, [ ] tr F and λ F, QF IM FFF F ( where Q( F map an arbtrary vetor to the pae orthogonal onto the pae panned by the olumn of F. 5. For gven et and, the relatve omplement of n denoted a. he ardnalty of a et denoted by. n 6. For a gven funton f ( x, t nth dervaton wth repet to x denoted by f ( x.

4 > Aepted for publaton n IEEE ranaton on Informaton heory 4 7. he lnear party regme defned by = β lm 0,. 8. he ublnear party regme defned by lm = he expreon f ( x = Ω ( g( x denote f( x g( x B. ytem Model Let a x for a ontant > 0. x, x,, x be jontly -pare vetor wth a upport et that belong to { { } } : = = =,,,, = =. hu, the number of nonzero oeffent of eah pare vetor, the nde of the nonzero oeffent of all the pare vetor are the ame and the nde belong to the upport et. In noy MMV wth dfferent enng matre, eah pare vetor ampled by t own enng matrx,.e., y = Fx + n =,, (3 where, all the enng matre have..d. Gauan element wth a zero mean and a unt varane, and all the noe vetor have..d. Gauan element wth a zero mean and a σ varane. We aume that all the noe vetor and all the enng matre are mutually ndependent. hen, we let x be the mallet nonzero magntude of all the pare vetor and R be the mum gnal-to-noe rato gven by C. Problem Formulaton R : = x σ. (4 We extend Akakaya and arokh [] deoder for noy MMV wth dfferent enng matre. It take all the meaurement vetor a t nput and yeld a upport et deon a t output It deon rule are gven n Defnton. { } ˆ Defnton : All the meaurement vetor { y, y,, y } and d : y, F, =,,,. and a et are δ jontly typal f the rank of ( Q( F y ( = A eah enng matrx ontan..d. Gauan element, the rank of eah F, =,,, M < Mδ. (5 F, =,,, wth a hgh probablty. he deon rule to fnd et that atfy (5 for all the gven meaurement vetor and δ > 0. In the entre paper, the upport et denoted by and any norret upport et denoted by, where ther ardnalte are,.e., = =. We defne the falure event, wheren the jont typal deoder fal to reontrut the orret upport et. Frt, ( Q( F = y : : = M Mδ (6 mple that the orret upport et not δ jontly typal wth all the meaurement vetor. ext, for any, ( Q( F = y : : = M < Mδ (7 mple that an norret upport et δ jontly typal wth all the meaurement vetor. Baed on thee falure event, we defne a falure probablty and gve t upper bound a follow:

5 > Aepted for publaton n IEEE ranaton on Informaton heory 5 p err { ˆ x x } : = :,, = : : : : + : {: } {: } where { } taken wth repet to all the noe vetor and { } taken wth repet to all the noe vetor and all the enng matre. We etablh Lemma and gven n Appendx A to gve upper bound of the probablte of the falure event. Combnng thee lemma wth (8 yeld where p defned n (3, d =, d Md ( M σ It of nteret to exae why { } n (6 the event Q F y, α p err { } { } + ( ( p( d, α p d + ( M σ + Md ( M α x =, and α = σ +. depend only on the noe vetor. A hown n Lemma 3, the random varable to defne, = where the meaurement vetor n (3 ont of the two part: the noe part (8 n and the gnal part Fx. he gnal part belong to the pae panned by the olumn of F. hen, a pefed n (, the orthogonal projeton matrx Q F map the meaurement vetor to the pae orthogonal onto the pae panned by the olumn of. hu, the random varable a funton of the noe vetor only. F III. MAI REUL A the man ontrbuton of th paper, th eton preent uffent and neeary ondton on M for a relable upport et reontruton n noy MMV wth dfferent enng matre. We nterpret the ondton to demontrate the beneft faltated by the jont party truture. A. uffent Condton on M In [8][], the author have hown that fewer meaurement M for a relable upport et reontruton are requred for noy MV n the lnear party regme, ompared to the ublnear party regme. Baed on the reult of [8][], we are motvated to exae, f the ame reult an be oberved n noy MMV wth dfferent enng matre. heorem : For any then the falure probablty ρ >, we let δ ρ err = M x. If the number of meaurement atfe M > + υ (9 p defned n (8 onverge to zero n the lnear party regme,.e., lm β ( 0, ( β log υ = > 0. ρ ρ log R + R + + Alo, under the ame ondton on ρ and δ, f the number of meaurement atfe =, where (0 M > + υ log ( then the falure probablty p defned n (8 onverge to zero n the ublnear party regme,.e., lm = 0, where err

6 > Aepted for publaton n IEEE ranaton on Informaton heory 6 υ = > 0. ρ ρ log R + R + + Proof: he proof gven n Appendx C. ( In term of,, and, the aymptot order of the uffent ondton on M for the lnear party regme Ω, wherea the order for the ublnear party regme Ω log. It onfrm that fewer meaurement are requred n the lnear party regme, ompared to the ublnear party regme. ext, from the uffent ondton, we oberve an nvere relatonhp between M and, owng to the jont party truture. h relatonhp mple that takng more meaurement vetor redue the number of requred meaurement M for a relable upport et reontruton. hen, the relatonhp an be ued for explanng the empral reult of Caone et al. [0] and Wu et al. [4]. In [0], the author have reported that the number of tranmtted bt per enor ould be nverely redued by the number of enor, whh mple that the tranmon ot of eah enor ould be aved. he reult an be onfrmed by our nvere relatonhp by onderng and M a the number of enor and the number of tranmtted bt per enor, repetvely. In [4], and M are ondered a the number of DI and the number of ample of eah DI, repetvely. Agan, t ha been oberved from [4] that the jont party truture enabled the number of ample of eah DI to be nverely redued by the number of DI, redung the aquton tme for eah DI. hee reult an be onfrmed by our nvere relatonhp. heorem : For any the falure probablty ρ >, we let δ ρ =, and be fxed. If the number of meaurement atfe M +, M x p err defned n (8 onverge to zero a takng nfntely many meaurement vetor,.e.,. Proof: he proof gven n Appendx C. heorem ugget that wth M +, a relable upport et reontruton for noy MMV wth dfferent enng matre poble by takng an nfnte number of meaurement vetor. A the mpat of noe an dappear n our uffent ondton, we beleve that the upport et reontruton beome robut agant noe by takng uffently large meaurement vetor. B. Duon on the uffent Condton We now exae the effet of R on the uffent ondton of heorem. he am to detere the relatonhp amongt, M and R for a relable upport et reontruton. Corollary : For any ρ >, we let δ ρ = M x. he uffent ondton of heorem are rewrtten a n the ublnear party regme,.e., lm = 0, and n the lnear party regme,.e., = β ( R + M > + 4log ρ ( R + M > + 4 ( log β ρ lm 0,. (3 (4 Proof: he proof gven n Appendx D. Corollary ugget that for a fxed M, a relable upport et reontruton poble by takng more meaurement vetor, although R low. amely, we oberve a noe reduton effet, whh how that ung the jont party truture lead to an nreae n R or a dereae n σ by the quare root of. h effet an explan the mprovement n the reontruton qualty of the DI, a emprally reported n [4]. We then mprove our noe reduton effet by onderng that Corollary : For any 3 = and 3. ρ >, we let δ ρ M x R larger than a ertan value. α = If

7 > Aepted for publaton n IEEE ranaton on Informaton heory 7 α ρ R =, ρ α ρ 3 (5 the uffent ondton of heorem are rewrtten a n the ublnear party regme,.e., lm = 0, and n the lnear party regme,.e., = β ( R + M > + ρ 4 log ( R + M > + 4 ( log β ρ lm 0,. (6 (7 Proof: he proof gven n Appendx D. Frt of all, Corollary requre ρ > 3 to enure that the lower bound n (5 potve. A mple omputaton how that Corollary requre fewer meaurement n both the regme ompared to Corollary beaue where the eond nequalty owng to ( + + R R ρ = ρ + R ρ t + R ρ = ( R + ρ = > for any ρ > 3 and t defned n (6. Bede, Corollary mprove the noe reduton effet oberved n Corollary by demontratng that R nreaed by for the regon of R gven n (5. heorem ugget, t to be noted, that M = + uffent for a relable upport et reontruton f uffently large wth a fxed and. hen, t would be nteretng to detere how large hould be requred for ahevng the mum number of meaurement at eah enor,.e., M = +. In wrele enor network [35], energy oure ued n enor are very lmted due to lmtaton of enor ze. hu, mzng the energy ued for tranmon of data at eah enor whh often lead to extendng the lfetme of the enor battery a value of mportane. h pont noted n Caone et. al. [0] a an advantage of ung dtrbuted ompreed enng on jont pare model- gnal enemble (ee eton V there. Corollary 3 whh am to provde a uffent ondton on for ahevng M = + thu motvated. Corollary 3: Let and be fxed and fnte. For any meaurement vetor atfe ( (( ε relable upport et reontruton poble,.e., p err ρ >, we let : = δ = ρ + x and M = +. If the number of > log + log max, log m> log m (((((((((((((((((( (8 < e for uffently mall ε ( 0,, where log µ and log µ are defned n (63 and (65, repetvely. he uffent ondton on dereang wth repet to Proof: he proof gven n Appendx D. R. o the bet of our knowledge, the uffent ondton on for a relable upport et reontruton have not yet been developed. A mlar reult ha been reported by ang and ehora [6], n whh they reported that M = Ω ( log and = uffe for a relable upport et reontruton n noy MMV wth the ame enng matrx, a uffently large. It of nteret to exae whether the uffent ondton * n (8 good. For th, we mplement the jont typal deoder n log log log

8 > Aepted for publaton n IEEE ranaton on Informaton heory 8 (5 and ondut experment for dfferent value of R and, for a fxed = 50. We ount the number of falure ourrene, wheren the jont typal deoder fal to reontrut the upport et. We obtan the mallet emp uh that the rato of the falure ourrene maller than ε = 0.0. By omparng emp wth * n (8, we ee that * approahe emp, a R uffently large. For example, we ee that emp = 8 and * = at R = 0 [db], =, and emp = 5 and * = 6 at R = 30 [db], =. A mlar trend oberved at = 5. For example, we ee that emp = and * = 9 at R = 0 [db], = 5, and emp = 7 and * = 0 at R = 30 [db], = 5. Flether et al.[9] have reported that the ML deoder requre M = + meaurement for a relable upport et reontruton n noy MV, when the gnal-to-noe rato uffently large. h reult an be oberved from Corollary 3. pefally, we aume that R uffently large for a fxed and. hen, from (63 and (65, t eay to ee that lm log m =, R lm log m = ρ log ρ. R Hene, (8 mplfed to ( ( > log + logε ρ log ρ. (9 ote that,, and ε are fxed. hu, for a large ρ, we have ( ( ρ log ρ log + log ε, (0 whh lead to. h reult ugget that the jont typal deoder requre M = + meaurement for a relable upport et reontruton n noy MV, whenever R uffently large and ρ atfe (0. C. eeary Condton on M We pefy a neeary ondton that mut be atfed by a deoder for a relable upport et reontruton n noy MMV wth dfferent enng matre. Unlke the uffent ondton of heorem, the neeary ondton preented for a fnte and. We begn by tranforg (3 nto y F x n = + y F x n M M = : y M = : F y = : x = : n ( where x an -pare vetor belongng to an nfnte et { x } x : = x x,, = where x the th element of x and the upport et of x. Owng to the jont party truture, the number of poble upport et (. hen, we defne a falure probablty a: p err F x x { ˆ xf} : = up :, ( where ˆ an etmate of the upport et baed on y and F n (. hen, Lemma III-3 of [0] yeld x x where ˆx an etmate for x baed on y and F n ( and { ˆ } xˆ { x } x { x } { ˆ } up xf, max x xxf, (3 { } : = x = x,, = x { } x whh a fnte et. Aume that x unformly dtrbuted over th fnte et. Applyng Fano nequalty [4] to (3 yeld

9 > Aepted for publaton n IEEE ranaton on Informaton heory 9 x { x} { xˆ x x F} { xˆ x F} max, ( xyf ; + log (4 log ( { } x where x and ˆx belong to the fnte et { x } xy ; the mutual nformaton between x and y. We get a neeary ondton on M to enure that the lower bound n (4 bounded away from zero, a follow: and heorem 3: Let and are fxed and fnte. In (, f the number of meaurement atfe then the falure probablty Proof: he proof gven n Appendx C. log log M < log ( + R p err defned n ( bounded away from zero. (5 IV. RELAIO O HE EXIIG IFORMAIO-HEOREIC REUL A. Relaton to oy MMV wth the ame enng Matrx [7] Jn and Rao [7] have exploted the Chernoff bound to obtan a tght uffent ondton on M for a relable upport et reontruton for noy MMV wth the ame enng matrx n the ublnear party regme. Owng to the omplated form of ther uffent ondton, they ould not learly how the beneft faltated by the jont party truture. hu, they mplfed ther ondton under enaro uh a: a low noe-level enaro and a enaro wth dental pare vetor. We ompare the uffent ondton ( wth thee ondton. Frt, n a low noe-level enaro, the uffent ondton [7] for noy MMV wth the ame enng matrx M log = Ω. (, (6 If <, the uffent ondton ( and (6 have the ame order, mplyng that there no gnfant performane gap n the upport et reontruton between the model. However, f >, (6 M ( log = Ω, wherea ( M ( log = Ω. It demontrate that noy MMV wth dfferent enng matre uperor to noy MMV wth the ame enng matrx or >, wth repet to M for a relable upport et reontruton. h omparon reult upport the perepton preented n eton I-C, wheren a more relable upport et reontruton ould be expeted n a noele MMV wth dfferent enng matre owng to the lnear ndependeny of the meaurement vetor. Moreover, t valdate the perepton, even n the preene of noe. eond, we onder a enaro wth dental pare vetor. hen, the uffent ondton of [7] M log = Ω. log ( + x σ (7 From (7, we oberve that σ redued by a fator of. However, the noe reduton effet for noy MMV wth the ame enng matrx requre a retrton, where all the pare vetor hould be dental, whh an be hardly aheved n prate. In ontrat, the noe reduton effet for noy MMV wth dfferent enng matre doe not requre th retrton, a hown n Corollare and. B. Relaton to oy MV [] Akakaya and arokh [] have ued the jont typal deoder to etablh the uffent ondton on M for a relable upport et reontruton n noy MV. hey exploted the exponental nequalte [33] to obtan the upper bound on the um of the weghted h-quare random varable. In th ubeton, we am to demontrate that the approahe developed n th paper are uperor to the ue of the exponental nequalte. hu, we ue the exponental nequalte to generalze ther bound for noy MMV wth dfferent enng matre. We gve Propoton and to prove that the generalzed bound are wore than the bound of Lemma and. Propoton : For any potve δ, we have

10 > Aepted for publaton n IEEE ranaton on Informaton heory 0 where both p( d and d are gven n Lemma, and { } p( d p,exp p δ M : = exp. 4σ M + δm σ,exp 4 (8 Proof: he proof gven n Appendx E. Propoton : For any and any δ > 0 uh that ( M x 0< δ <, (9, we have where both (, R { } p d λ p,,exp p( d, λ and d R, λ ( R are gven n Lemma and p ( M Mδ : = exp x 4 α M =,,,exp, (30 and α, defned n (39 and x, defned n (43. Proof: he proof gven n Appendx E. If =, we an ee that p,exp and p,,exp are equvalent to the bound of Akakaya and arokh []. Propoton and tate that the bound on the falure probablty of Lemma and are tghter than the bound of [] for noy MV. V. COCLUIO We have tuded a upport et reontruton problem for C wth noy MMV wth dfferent enng matre. he unon and Chernoff bound have been ued to obtan the upper bound of the falure probablty of the upport et reontruton, and Fano nequalty ha been ued to obtan the lower bound of th falure probablty. A we have obtaned the upper bound by analyzng an exhautve earh deoder, the bound ued to meaure the performane of C wth noy MMV wth dfferent enng matre. We have then developed the neeary and uffent ondton n term of the party, the ambent dmenon, the number of meaurement M, the number of meaurement vetor, and the mum gnal-to-noe rato R. hee ondton an be ued a gudelne for deterng the ytem parameter n varou C applaton wth noy MMV wth dfferent enng matre. he ondton are nterpreted to provde theoretal explanaton for the beneft faltated by the jont party truture n noy MMV wth dfferent enng matre:. From the uffent ondton of heorem, we have oberved the nvere relatonhp between M and. Owng to the nvere relaton, we an take fewer meaurement M per eah meaurement vetor for a relable upport et reontruton by takng more meaurement vetor.. From the uffent ondton of Corollare and, we have oberved the noe reduton effet, whh how that the uage of the jont party truture reult n an nreae n R or a dereae n σ by a fator of. herefore, the upport et reontruton an be robut agant noe by takng uffently large meaurement vetor.. From heorem, we have hown that M = + aheved for a fxed and, a uffently large. From Corollary 3, we have provded the uffent ondton on to reontrut the upport et for a fxed,, and M = +. he above theoretal explanaton onfrm the empral beneft of the jont party truture, a hown n C applaton wth noy MMV wth dfferent enng matre [0][4]. We have ompared our uffent ondton for noy MMV wth dfferent enng matre wth the other extng reult [7] for noy MMV wth the ame enng matrx. For a low-level noe enaro wth, we have hown that the number of meaurement for a relable upport et reontruton for noy MMV wth dfferent enng matre leer than that for noy MMV wth the ame enng matrx. Alo, we have oberved that noy MMV wth dfferent enng matre enjoy the noe reduton effet for arbtrary jontly pare vetor, wherea, th noe reduton ext n noy MMV wth the ame enng matrx, only f all the pare vetor are dental.

11 > Aepted for publaton n IEEE ranaton on Informaton heory APPEDIX A LEMMA AD h eton preent Lemma and, whh gve upper bound of the probablte of the falure event defned n (6 and (7, repetvely. Alo, for mplty, we defne ( ( M M p x = exp x x +. (3 Lemma : For any potve δ, we have where the funton p defned n (3, and { } ( M exp d + d = p d ( M ( M Md d : = > 0. σ (3 (33 Proof: From (6, we have { } = { Z W} + { Z W} (34 where Z defned n Lemma 3, and δ σ W = M + M, =,. Applyng the Chernoff bound [3] to (34 yeld {: } exp( tw exp( tz : = ( M exp( tw ( t = ((( (((((( = = : f ( t ; W (35 where the equalty from Lemma 3, 0 mzer of f t ; W, where hu, f ( t; W f ( t ; W where p( d and hu, t < and t ( 0,. A eah ( ; ( t W M =, =,. for eah. If W 0, t lear that { W } { } { Z W} f ( t = ; W = p( d f t W onvex, t = t at ( Z = 0 beaue Z quadrat. hu, f t ; W = 0 yeld the (36 d are defned n (3 and (33, repetvely. If W > 0 then f ( t; W f ( t; W Fnally, ombnng (36 and (37 lead to (3. ( f ( t W log f t ; W log ; Lemma : Let and a matrx R be = M d+ log d log + d < 0. { } = f ( t; W + f ( t; W f ( t; W ( ( M M = exp d + d. beaue (37

12 > Aepted for publaton n IEEE ranaton on Informaton heory where R α,i M = (38 α, IM α, : = + x> > 0. (39 Conder any potve δ uh that where λ ( R the mallet egenvalue of R. hen, ( ( R 0< δ < M λ σ where the funton p defned n (3, { } d, λ ( ( M M exp ( d, λ R d, λ ( R = p( d, λ R p d (, α ( M σ + Md : = ( 0,, R M λ R (40 (4 α : = σ + x, (4 and = x\ {,,, } x = : x, (43 Proof: From (7, we have { } = { Z < W} { Z < W} { Z < W} (44 where Z defned n Lemma 4, and σ W = M Mδ, =,. (45 Applyng the Chernoff bound [3] to (44 yeld for t < 0, {: } exp( tw exp( tz ( M = exp( tw ( tλ R = ( M exp( tw ( tλ ( R = : f( tw ;. : (46 where the equalty from Lemma 4 and the eond nequalty due to that all the egenvalue are potve. We then defne a h t : = log f tw ;. hen, funton ( λ ( R λ R h t = M t > 0 whh mple that h onvex wth repet to t. It lead to that f n (46 logarthmally onvex. hu t yeld the mzer of ( ; f tw where = t at ( f tw ; = 0

13 > Aepted for publaton n IEEE ranaton on Informaton heory 3 ( λ ( R t W M = < 0. ubttutng t n (46 yeld { } f ( t ; W ( ( M M = exp ( d, λ R d, λ ( R = p( d,. λ R (47 where d, λ ( R defned n (4 and p defned n (3. ext, let β ( M and Due to (48 and (49, = and x = n the upper bound (47. hen, we have β p x x ( β ( x R d, λ p x x ( R ( β β = βx exp x x > 0 = exp, where (48 x = x < 0. (49 λ R p x p x x = λ x λ ( R ( β βx exp x x 0 β = < whh how that the upper bound n (47 dereang wth repet to λ ( R. hen, red that the matrx n (38 the ovarane matrx of a multvarate Gauan vetor b n (58. hen for any norret upport et, t mallet egenvalue an be ealy omputed and lower bounded by ( R {,,, } λ = α, = + x, α (50 where x, defned n (43 and α defned n (4. hu, for any norret upport et, we onlude that whh omplete the proof. p( d, R, α { } p d λ APPEDIX B LEMMA 3 AD 4 Frt of all, we gve the harf theorem [34] to ompute the moment generatng funton of a quadrat random varable. We then make Lemma 3 and 4 to gve the moment generatng funton of the random varable of and that were ued n the proof of Lemma and, repetvely. harf' theorem [Page 64, [34]]: Let b random varable Q ( bm ( bm quadrat wth [ Q ] = tr [ R], [ ] tr Lemma 3: In (6, defne a quadrat random varable be a multvarate Gauan vetor wth a mean m and a ovarane R. hen a ( tq = ( tλ exp. R = Z : = Q F y. = Q RR = and for any t (5

14 > Aepted for publaton n IEEE ranaton on Informaton heory 4 and for any 0 < t < 0.5, hen, [ Z ] = ( M, [ Z ] = ( M Proof: he orthogonal projeton matrx deompoed a where = ( ( M exp tz t. (5 = Q F U D U D a dagonal matrx, whoe frt M dagonal are one and the reman are zero, and U a untary matrx. hen, = Q F = y = Q F n = Z ( = D U n = Dw (( ( ( = = = : w (53 where w a multvarate Gauan vetor wth mean dagonal matrx are one, we have whh quadrat, where 0 and ovarane I. ne the frt M dagonal element of eah M M M Z = Dw = w = = = = = w w = ww (54 and w = w w w M w = w w w. (55 In (53, w detered by U and. n ne the element of U and n are ndependent, w and j w are mutually ndependent for any j. he ovarane matrx of w an dentty matrx. hu, applyng the harf theorem to Z omplete the proof. Lemma 4: In (7, for any, defne a quadrat random varable hen, Z = tr R, Z = tr R R and for any t, where R gven n (38. Proof: mlar to the proof of Lemma 3, where = Z : = Q F y. (56 M ( t ( tλ = = exp Z R, = Q F U D U D a dagonal matrx, whoe frt M dagonal are one and the reman are zero, and U a untary matrx. hen,

15 > Aepted for publaton n IEEE ranaton on Informaton heory 5 Z = Q F = y = Q F = ( = D U = Db ( ( = = = : b (57 where b a multvarate Gauan vetor wth mean 0 M and ( = + b x I M and n f x ( u. = + u whh quadrat, where and u ne the frt M dagonal element of eah dagonal matrx are one, we have Z b ( M = Db = = = = = = b b = bb = b b b M b (58 ( b = b b b In (57, b detered by U, n and{ fu : u }. ne the element of U, n. and { u : u } f are ndependent, j and b are mutually ndependent for any j. he ovarane matrx of b dagonal a hown n (38. hu, applyng the harf theorem to Z omplete the proof. APPEDIX C PROOF OF HEOREM, AD 3 Proof of heorem : It lear that goe to nfnty a goe to nfnty n the lnear party regme. hen, let M = where >. From (3, {: } ( ( ( d d log : log + + log (((( = : A b where A < 0 due to (33. hu, { } ( ( A lm lm exp + log = 0 mplyng that the probablty that the orret upport et not δ jontly typal wth all the meaurement vetor vanhe. ext, from (40, : {: } (( p( d, α log log ( ( ( ( t t = log + log + (( ( ( ( log ( g + + (((((((( = : η = : g (59 where the lat nequalty due to e exp log. (60

16 > Aepted for publaton n IEEE ranaton on Informaton heory 6 In (59, g < 0 for any t where ρ t = ( 0,. + R (6 If > + υ, then η < 0, whh yeld { } ( η lm lm exp = 0 mplyng that the probablty that all norret upport et are δ jontly typal wth all the meaurement vetor vanhe. hu the falure probablty p defned n (8 onverge to zero f M atfe (9. err ext, the reman to derve ( n the ublnear party regme. mlarly, let {: } ( ( d d log : log log + + log (((( = : A M = + log wher >. From (3, where A < 0 due to (33. hu, { } ( A lm lm exp log + log = 0 mplyng that the probablty that the orret upport et not δ jontly typal wth all the meaurement vetor vanhe. hen, from (40, : {: } (( p( d, α log log ( ( ( t t = log + log + log (( ( ( ( g = : η = : g + log + (( where the lat nequalty due to the bound n (60 and g < 0 for any t n (6. If > υ, then η < 0, whh yeld { } ( η lm lm exp log + = 0 mplyng that the probablty that all norret upport et are δ jontly typal wth all the meaurement vetor vanhe. hu, the falure probablty p defned n (8 onverge to zero f M atfe (, whh omplete the proof. err Proof of heorem : From Lemma, M M :{: } exp d ( d +. (6 (((((((((( = : µ If M +, we have ( M ( ( d d log µ = log + < 0 (63 due to (33, whh mple µ <. From Lemma, mlarly, f M +, we have M M :{: } exp ( d, d. α (64, α (((((((((((( = : µ

17 > Aepted for publaton n IEEE ranaton on Informaton heory 7 due to (6, whh mple µ <. hu, we onlude for M + whh omplete the proof. ( M ( ( t t log log 0 µ = + < (65 ( lm p lm m + lm m = 0 err Proof of heorem 3: he mutual nformaton n (4 bounded by ( xyf ; = h( yf h( yxf, h( y h( n M h( y h n = ( + ( ( π σ ( π σ M log e x log e = + ( M log R where h( x the dfferental entropy of x, and h( xy the ondtonal entropy of x gven y. he lat nequalty due to that the Gauan dtrbuton maxmze the dfferental entropy. he denoator n (4 bounded by for uffently large. hen, ( { x } log = log > log p err F x M x F x ˆ M { x} x M{ x} { > ˆ > xf} = up, { xˆ xxf} max, ( M log + R + log >. log (66 From (66, the falure probablty bounded away from zero by zero f (5 atfed, whh omplete the proof. APPEDIX D PROOF OF COROLLARIE, AD 3 x Proof of Corollary : From the nequalty log ( + for x ( ] x + x, 0, 4 t 4 υ = < < log t + t t t (67 where t defned n (6. hen, From (6, υ 4 <. (68 t t = ρ ( R +. (69 Combnng (, (68 and (69 lead to (3. h approah ued to get (4 ung the followng equalty where β υ = υ log β (70 lm = 0,, whh omplete the proof. Proof of Corollary : ubttutng α = n (5, and rearrangng the reult wth repet to t an yeld t <, where t 3 3

18 > Aepted for publaton n IEEE ranaton on Informaton heory 8 defned n (6. hen from (67, a mple omputaton yeld that whh mmedately yeld that 4 t 4 υ < t t υ 4 <. (7 t where t = ρ ( R +. (7 Combnng (, (7 and (7 lead to (6. h approah ued to get (7 ung (70, whh omplete the proof. Proof of Corollary 3: We aume that µ µ and { } { } (( p + + µ < e <. (73 err hen, f the number of meaurement vetor atfe logε log (( + > > 0, log µ (73 aheved for mall ε, and hene, relable upport et reontruton poble. If µ < µ, (73 and (74 by replang µ by µ, where logε log (( + > > 0. log µ > (74 we obtan nequalte mlar to (75 Combnng (74 and (75 yeld (8. ext, a mple omputaton yeld that for any d n (33, log µ d d + = < ( d 0 where log µ gven n (63. From (33, we ee d R that lead to log m R. Alo, for any t n (6, log µ t t = < 0 ( t where log µ gven n (65. From (6, we ee t R that lead to log m R. Hene, the uffent ondton on n (8 turn out to be a dereang funton wth repet to R, whh omplete the proof. APPEDIX E PROOF OF PROPOIIO AD Frt of all, we ntrodue the exponental nequalte [33], and ue them n the proof of Propoton and. he exponental nequalte [33]: Let Y, =,,, D be..d. Gauan varable wth a zero mean and a unt varane. hen, let α, =,, D be non-negatve. We et and let D up, = α = α α = α

19 > Aepted for publaton n IEEE ranaton on Informaton heory 9 hen, the followng nequalte hold for any potve x ( Y D α. = (76 Y = { Y α x + α x} exp ( x (77 { Y α x} exp ( x Proof of Propoton : In the proof of Lemma 3, Z repreented by where w. (78 Z M = = = w ( Gauan wth a zero mean and a unt varane. Defne a random varable Y a whh of the form of (76. hen, Combnng A wth (78 gve and ombnng B wth (77 gve Z M w ( Y = M = = =. {: } = { Y Mδ σ } + { Y Mδ σ } :. = : A = : B { } { δ σ } = ( : Y M : Y M x M δ exp 4 4( M σ (((((( = : C { } { δ σ } Y M = Y M x + x p,exp where and p defned n (8. It readly een that p,exp C,,exp ext, from (3 and (8, where d > 0 defned n (33. hen, we have whh lead to { } p,exp = ( ( ( + log p d M log d d log p = M d + 4d,exp. ( ( = ( ( + d d+ d ( + d p d M log log 4. p ((((((((((((,exp = : g( d For any d > 0, omplete the proof. = ( + 3 ( + ( + < 0 and g ( d g( d d d d d d max = 0. hu, we onlude d > 0 pd log 0,,exp p whh Proof of Propoton : In the proof of Lemma 4, Z repreented by

20 > Aepted for publaton n IEEE ranaton on Informaton heory 0 where, α defned n (39 and g whh of the form of (76. hen, from (44 Z = = M b ( = = M α, g = = ( Gauan wth a zero mean and a unt varane. Defne a new random varable Y a Z α = =, Y = M M ( g ( =. { x = } {: } Y < Mδ ( M : : : Y < Mδ ( M x, (((((((( = : A p,,exp where p,,exp defned n (30, the lat nequalty due to (78. Due to (9, A negatve. hu the exponental nequalty of (78 gve the upper bound p.,,exp ext, from (40 and (30, and log ( p d =, ( M t+ log ( t l R log p Mδ x ( M, M,,exp 4 x, + σ = 4 M t where t ( 0,, defned n (6 and the nequalty due to (50. hen, (79 For any t ( 0,, p( d, l R ( M log ( t + t + log ( t. p,,exp 4 (((((( = : gt gt = < 0 and t t t, l ( R max g t = 0. hu, we onlude log 0, pd p,,exp whh omplete the proof. REFERECE [] C. E. hannon, A mathematal theory of ommunaton, he Bell ytem ehnal Journal, vol. 7, pp , , Jul. Ot [] D. Donoho, Compreed enng, IEEE ran. Inf. heory, vol. 5, no. 4, pp , Apr [3] E. J. Cande and. ao, Deodng by Lnear Programg, IEEE ran. Inf. heory, vol. 5, no., pp , De [4] E. J. Cande, J. Romberg, and. ao, "Robut unertanty prnple: Exat gnal reovery from hghly nomplete frequeny nformaton," IEEE ran. Inf. heory, vol. 5, no., pp , Feb [5] E. J. Cande and. ao, ear optmal gnal reovery from random projeton : Unveral enodng tratege, IEEE ran. Inf. heory, vol. 5, no., pp , De [6] E. J. Cande and. ao, table gnal reovery from nomplete and naurate meaurement, Comm. Pure Appl. Math., vol. 59, no. 8, pp. 07 3, Aug [7] E. J. Cande and M. Wakn, An ntroduton to ompreve enng, IEEE g. Proe. Mag., vol. 5, no., pp. 30, Mar [8] D. Baron, M. F. Duarte, M. B. Wakn,. arvotham, and R. G. Baranuk, Dtrbuted ompreed enng, Arxv preprnt arxv: , 009. [9] M. F. Duarte,. arvotham, D. Baron, M. B. Wakn and R. G. Baranuk, Dtrbuted ompreed enng of jontly pare gnal, Alomar Conferene on gnal, ytem & omputer, Paf Grove, CA, UA, ov. 005, pp [0] C. Caone, D. Brunell, and L. Benn, Compreve enng optmzaton for gnal enemble n W, IEEE ran. on Indutral Informat, vol. 0, no., pp , Feb. 04.

21 > Aepted for publaton n IEEE ranaton on Informaton heory [] W. Chen, R. D. Rodrgue, and I. J. Waell, Dtrbuted ompreve enng reontruton va ommon upport dovery, IEEE Internatonal Conferene on Communaton (ICC, yoto, Japan, Jun. 0, pp. -5. [] J. Olver, WoongB Lee, and Heung-o Lee, Flter wth random tranmttane for mprovng reoluton n flter-array-baed petrometer, Opt Expre, vol., no. 4, pp , Feb. 03. [3]. Gognnen and A. ehora, arget etmaton ung pare modelng for dtrbuted MIMO radar, IEEE ran. gnal Proe., vol. 59, no., pp , ov. 0. [4] Y. Wu, Y. Zhu, Q. ang, C. Zou, W. Lu, R. Da, X. Lu, E. X. Xu, L. Yng, and D. Lang, Aelerated MR dffuon tenor magng ung dtrbuted ompreed enng, Magnet Reonane n Medne, vol. 7, no., pp , 04. [5] M. Elad, pare and Redundant Repreentaton: From heory to Applaton n gnal and Image Proeng, prnger, 00. [6] Y. C. Eldar and G. utynok, Compreed enng: heory and Applaton, Cambrdge Unverty Pre, May. 0. [7] Y. C. Eldar, amplng heory: Beyond Bandlmted ytem, Cambrdge Unverty Pre, Apr. 05. [8] M. J. Wanwrght, Informaton-theoret lmt on party reovery n the hgh-dmenonal and noy ettng, IEEE ran. Inf. heory, vol. 55, no., pp , De [9] A.. Flether,. Rangan and V. Goyal, eeary and uffent ondton for party pattern reovery, IEEE ran. Inf. heory, vol. 55, no., pp , De [0]. Aeron, V. algrama, and M. Zhao, Informaton-theoret bound for ompreed enng, IEEE ran. Inf. heory, vol. 56, no. 0, pp , Ot. 00. [] M. Akakaya and V. arokh, hannon-theoret lmt on noy ompreve amplng, IEEE ran. Inf. heory, vol. 56, no., pp , Jan. 00. [] J. arlett, J. Evan and. Dey, Compreed enng wth pror nformaton: nformaton-theoret lmt and pratal deoder, IEEE ran. gnal Proe., vol. 6, no., pp , Jan. 03. [3] J. arlett and V. Cevher, Lmt on upport reovery wth probablt model: an nformaton-theoret framework, Arxv preprnt arxv: , 05. [4]. Cover and J. homa, Element of Informaton heory, ew York, UA: Wley, 006. [5] J. Chen and X. Huo, heoretal reult on pare repreentaton of multple-meaurement vetor, IEEE ran. gnal Proe., vol. 56, no., pp , De [6] G. ang and A. ehora, Performane analy for pare upport reovery, IEEE ran. Inf. heory, vol. 56, no. 3, pp , Mar. 00. [7] Y. Jn and B. D. Rao, upport reovery of pare gnal n the preene of multple meaurement Vetor, IEEE ran. Inf. heory, vol. 59, no. 5, pp , May. 03. [8] M. F. Duarte, M. B. Wakn, D. Braon,. arvotham, and R. G. Baranuk, Meaurement bound for pare gnal enemble va graphal model, IEEE ran. Inf. heory, vol. 59, no. 7, pp , Jul. 03. [9] J. m, O. Lee, and J. Ye, Compreve MUIC: revtng the lnk between ompreve enng and array gnal proeng, IEEE ran. Inf. heory, vol. 58, no., pp , Jan. 0. [30] J. D. Blanhard and M. E. Dave, Reovery guarantee for rank aware purut, IEEE gnal Proeng Lett., vol. 9, no. 7, pp , Jul. 0. [3] J. Zhu, D. Baron, and F. rzakala, Performane lmt for noy mult-meaurement vetor problem, Arxv preprnt ArXv: v, 06. [3] H. L. Van ree, Deteton, Etmaton, and Modulaton heory, Part I, ew York, UA: Wley-Interene, 00. [33] B. Laurent and P. Maart, Adaptve etmaton of a quadrat funtonal by model eleton, Ann. tatt., vol. 8, no. 5, pp , Ot [34] L. L. harf, tattal gnal Proeng: Deteton, Etmaton, and me ere Analy, Addon-Weley Publhng Company, 99. [35] R. Rajagopalan and P.. Varheny, Data-aggregaton tehnque n enor network: A urvey, IEEE Commun. urvey utor., vol. 8, no. 4, pp , Fourth Quarter 006.