Single-letter Characterization of Signal Estimation from Linear Measurements Dongning Guo Dror Baron Shlomo Shamai The work has been supported by the European Commission in the framework of the FP7 Network of Excellence in Wireless Communications NEWCOM++, by the Israel Science Foundation, and by the National Science Foundation.
Linear Measurement Systems 1809: Theoria motus corporum coelestium Gauss introduced application of least squares (regression) to solve noisy linear systems motivated by astronomy/navigation Goal: estimate input x to explain measurements y
Non-linear Signal Estimation Linear signal estimation (least squares) sub-optimal example: hard decisions used to estimate binary data Difficult problem with noisy observations even over-determined problems can be challenging Need information theoretic framework for non-linear signal estimation in linear measurement systems underdetermined overerdetermined
Linear Measurement Application Areas Compressed sensing Multiuser communication (CDMA) Medical imaging (tomography) Financial prediction Electromagnetic scattering Seismic imaging (oil industry)
Problem Definition
Setting Replace samples by more general measurements based on a few linear projections (inner products) measurements sparse signal # non-zeros
Signal Model Signal entry X n = B n U n iid B n» Bernoulli(ε) sparse iid U n» P U P X Bernoulli(ε) Multiplier P U
Non-Sparse Input Can use ε=1 X n = U n P U
Measurement Noise Measurement process is typically analog Analog systems add noise, non-linearities, etc. Assume Gaussian noise for ease of analysis Can be generalized to non-gaussian noise [Guo & Wang 2007; Rangan 2010]
Noise Model Noiseless measurements denoted y 0 Noise Noisy measurements Unit-norm columns SNR=γ noiseless SNR
Allerton 2006 [Sarvotham, Baron, & Baraniuk] Model process as measurement channel source encoder channel encoder channel channel decoder source decoder CS measurement CS decoding Measurements provide information! Preliminary single-letter bound for compressed sensing and linear measurement systems
Numerous single-letter bounds [Aeron, Zhao, & Saligrama] [Akcakaya and Tarokh] [Rangan, Fletcher, & Goyal] [Gastpar & Reeves] [Wang, Wainwright, & Ramchandran] [Tune, Bhaskaran, & Hanly] Related Results BP Multiuser detection [Tanaka & Takeda] [Guo & Wang] [Montanari & Tse] Arbitrary noise [Rangan] [Guo & Wang]
Goal: Precise Single-letter Characterization of Optimal CS [Guo, Baron, & Shamai 2009]
What Single-letter Characterization? Φ,Φ channel posterior Ultimately what can one say about X n given Y? (sufficient statistic) Very complicated Want a simple characterization of its quality Large-system limit:
Main Result: Single-letter Characterization Φ,Φ Result1: Conditioned on X n =x n, the observations (Y,Φ) are statistically equivalent to channel posterior η easy to compute degradation Estimation quality from (Y,Φ) just as good as noisier scalar observation
Details η2(0,1) is fixed point of Take-home point: degraded scalar channel Non-rigorous owing to replica method w/ symmetry assumption used in CDMA detection [Tanaka 2002, Guo & Verdu 2005] Related analysis [Rangan, Fletcher, & Goyal 2009] MMSE estimate (not posterior) using [Guo & Verdu 2005] extended to several CS algorithms particularly LASSO
Decoupling [Guo, Baron, & Shamai 2009]
Decoupling Result Result2: Large system limit; any arbitrary (constant) L input elements decouple: Take-home point: interference from each individual signal entry vanishes
Sparse Measurement Matrices [Sarvotham, Baron, & Baraniuk 2006] [Guo, Baron, & Shamai 2009] [Baron, Sarvotham, & Baraniuk 2010]
Sparse Measurement Matrices LDPC measurement matrix (sparse) Mostly zeros in Φ; nonzeros» P Φ Each row contains ¼Nq randomly placed nonzeros Fast matrix-vector multiplication fast encoding / decoding sparse matrix
CS Decoding Using BP [Baron, Sarvotham, & Baraniuk 2006] Measurement matrix represented by graph Estimate input iteratively Implemented via nonparametric BP [Bickson,Sommer, ] signal x measurements y
Identical Single-letter Characterization w/bp [Montanari & Tse 2006; Guo & Wang 2008] Result3: Conditioned on X n =x n, the observations (Y,Φ) are statistically equivalent to Rigorous result identical degradation Sparse matrices just as good BP is asymptotically optimal!
CS-BP vs Other CS Methods (N=1000, ε=0.1, q=0.02) 90 80 MMSE 70 MMSE 60 50 M 40
Conclusion Single-letter characterization of CS Decoupling Sparse matrices just as good Asymptotically optimal CS-BP algorithm
THE END