Compressive Sensing of Sparse Tensor
|
|
- Everett Paul
- 5 years ago
- Views:
Transcription
1 Shmuel Friedland Univ. Illinois at Chicago Matheon Workshop CSA2013 December 12, 2013 joint work with Q. Li and D. Schonfeld, UIC
2 Abstract Conventional Compressive sensing (CS) theory relies on data representation in the form of vectors. Many data types in various applications such as color imaging, video sequences, and multi-sensor networks, are intrinsically represented by higher-order tensors. We propose Generalized Tensor Compressive Sensing (GTCS) a unified framework for compressive sensing of higher-order spare tensors. Similar to Caiafa-Cichocki GTCS offers an efficient means for representation of multidimensional data by providing simultaneous acquisition and compression from all tensor modes. We compare the performance of the proposed method with Kronecker compressive sensing (KCS, Duarte-Baraniuk), and multi-way compressive sensing (MWCS, Sidiropoulus-Kyrillidis). We demonstrate experimentally that GTCS outperforms KCS and MWCS in terms of both accuracy and speed.
3 Overview CS of vectors CS of matrices Simulations of CS for matrices CS of tensors Simulations of CS for tensors Conclusions
4 Compressive sensing of vectors: Noiseless Σ s,n is the set of all x R N with at most s nonzero coordinates Sparse version of CS: Given x Σ s,n compress it to a short vector y = (y 1,..., y M ), M << N and send it to receiver receiver gets y, possible with noise, decodes to x Compressible version: coordinates of x have fast power law decay Solution: y = Ax, A R M N a specially chosen matrix, e.g. s-n. p. Sparse noiseless recovery: x = arg min{ z 1, Az = y} A has s-null property if for each Aw = 0, w 0, w 1 > 2 w S 1 S [N] := {1,..., N}, S = s, w S has zero coordinates outside S and coincides with w on S Recovery condition M cs log(n/s), noiseless reconstruction O(N 3 )
5 Compressive sensing of vectors with noise A R M N satisfies restricted isometry property (RIP s ): (1 δ s ) x 2 Ax 2 (1 + δ s ) x 2 for all x Σ s,n and for some δ s (0, 1) Recover with noise: ˆx = arg min{ z 1, Az y} 2 ε} reconstruction O(N 3 ) THM: Assume that A satisfies RIP 2s property with δ 2s (0, 2 1). Let x Σ s,n, y = Ax + e, e 2 ε. Then ˆx x 2 C 2 ε, where C 2 = 4 1+δ 2s 1 (1+ 2)δ 2s
6 Compressive sensing of matrices I - noiseless X = [x ij ] = [x 1... x N1 ] R N 1 N 2 is s-sparse. Y = U 1 XU 2 = [y 1,..., y M2 ] R M 1 M 2, U 1 R M 1 N 1, U 2 = R M 2 N 2 M i cs log(n i /s) and U i has s-null property for i = 1, 2 Thm M: X is determined from noiseless Y. Algo 1: Z = [z 1... z M2 ] = XU 2 RN 1 M 2 each z i a linear combination of columns of X hence s-sparse Y = U 1 Z = [U 1 z 1,..., U 1 z M2 ] so y i = U 1 z i for i [M 2 ] Recover each z i to obtain Z Cost: M 2 O(N 3 1 ) = O((log N 2)N 3 1 ) Z = U 2 X = [U 2 x 1... U 2 x N1 ] Recover each x i from i th column of Z Cost: N 1 O(N 3 2 ) = O(N 1N 3 2 ), Total cost: O(N 1N (log N 2)N 3 1 )
7 Compressive sensing of matrices II - noiseless Algo 2: Decompose Y = r i=1 u iv i, u 1,..., u r, v 1,..., v r span column and row spaces of Y respectively for example a rank decomposition of Y : r = rank Y Claim u i = U 1 a i, v j = U 2 b j, a i, b j are s-sparse, i, j [r]. Find a i, b j. Then X = r i=1 a ib i Explanation: Each vector in column and row spaces of X is s-sparse: Range(Y ) = U 1 Range(X), Range(Y ) = U 2 Range(X ) Cost: Rank decomposition: O(rM 1 M 2 ) using Gauss elimination or SVD Note: rank Y rank X s Reconstructions of a i, b j : O(r(N N3 2 )) Reconstruction of X: O(rs 2 ) Maximal cost: O(s max(n 1, N 2 ) 3 )
8 Why algorithm 2 works Claim 1: Every vector in Range X and Range X is s-sparse. Claim 2: Let X 1 = r i=1 a ib i. Then X = X 1. Prf: Assume 0 X X 1 = k j=1 c jd j, c 1,..., c k & d 1,..., d k lin. ind. as Range X 1 Range X, Range X 1 Range X c 1,..., c k Range X, d 1,..., d k Range X Claim: U 1 c 1,... U 1 c k lin.ind.. Suppose 0 = k j=1 t iu 1 c j = U 1 k j=1 t jc j. As c := k j=1 t jc j Range X, c is s-sparse. As U 1 has null s-property c = 0 t 1 =... = t k = 0. 0 = Y Y = U 1 (X X 1 )U 2 = k j=1 (U 1c j )(d j U 2 ) U 2 d 1 =... = U 2 d k = 0 d 1 =... d k = 0 as each d i is s-sparse So X X 1 = 0 contradition
9 Sum.-Noiseless CS of matrices & vectors as matrices 1. Both algorithms are highly parallelizable 2. Algorithm 2 is faster by factor s min(n 1, N 2 ) at least 3. In many instances but not all algorithm 1 performs better. 4. Caveat: the compression is : M 1 M 2 C 2 (log N 1 )(log N 2 ). 5. Converting vector of length N to a matrix Assuming N 1 = N α, N 2 = N 1 α the cost of vector compressing is O(N 3 ) the cost of algorithm 1 is O((log N)N 9 5 ), α = 3 5 the cost of algorithm 2 is O(sN 3 2 ), α = 1 2, s = O(log N)(?) Remark 1: The cost of computing Y from s-sparse X: 2sM 1 M 2 (Decompose X as sum of s standard rank one matrices)
10 Compressive sensing of matrices with noise - I Y = U 1 XU 2 + E, E F ε Algo 1: Recover each z i by ẑ i = arg min{ w i 1, U 1 w i c i (Y )} 2 ε} Form Ẑ = [z 1... z M2 ] R N 1 m 2 c i (Ẑ ) c i(xu 2 2 C 2 ε (optimistically C 2ε M ) Ẑ XU 2 F MC 2 ε (optimistically C 2 ε) Obtain ˆX by recovering each row of X: ˆb i = arg min{ w i 1, U 2 w i c i (Ẑ )} 2 MC 2 ε} ˆb i c i (X ) 2 Mε (optimistically ˆb i c i (X ) 2 ε Variation: Estimate s and take best rank s-approximation of Y : Y s Similarly, after computing Ẑ from Y s replace Ẑ by Z s. Costlier and no estimates
11 Numerical simulations We experimentally demonstrate the performance of GTCS methods on sparse and compressible images and video sequences. Our benchmark algorithm is Duarte-Baraniuk 2010 named Kronecker compressive sensing (KCS) Another method is multi-way compressed sensing of Sidoropoulus-Kyrillidis (MWCS) 2012 Our experiments use the l 1 -minimization solvers of Candes-Romberg. We set the same threshold to determine the termination of l 1 -minimization in all subsequent experiments. All simulations are executed on a desktop with 2.4 GHz Intel Core i5 CPU and 8GB RAM. We set M i = K
12 UIC logo (a) The original sparse image (b) GTCS-S recovered image recov- (d) KCS image (c) GTCS-P ered image recovered
13 PSNR and reconstruction times for UIC logo PSNR (db) GTCS S GTCS P KCS Reconstruction time (sec) GTCS S GTCS P KCS Normalized number of samples (K*K/N) (a) PSNR comparison Normalized number of samples (K*K/N) (b) Recovery time comparison Figure : PSNR and reconstruction time comparison on sparse image.
14 Explanation of UIC logo representation I The original UIC black and white image is of size (N = 4096 pixels). Its columns are 14-sparse and rows are 18-sparse. The image itself is 178-sparse. For each mode, the randomly constructed Gaussian matrix U is of size K 64. So KCS measurement matrix U U is of size K The total number of samples is K 2. The normalized number of samples is K 2 N. In the matrix case, GTCS-P coincides with MWCS and we simply conduct SVD on the compressed image in the decomposition stage of GTCS-P. We comprehensively examine the performance of all the above methods by varying K from 1 to 45.
15 Explanation of UIC logo representation II Figure?? and?? compare the peak signal to noise ratio (PSNR) and the recovery time respectively. Both KCS and GTCS methods achieve PSNR over 30dB when K = 39. As K increases, GTCS-S tends to outperform KCS in terms of both accuracy and efficiency. Although PSNR of GTCS-P is the lowest among the three methods, it is most time efficient. Moreover, with parallelization of GTCS-P, the recovery procedure can be further accelerated considerably. The reconstructed images when K = 38, that is, using 0.35 normalized number of samples, are shown in Figure??????. Though GTCS-P usually recovers much noisier image, it is good at recovering the non-zero structure of the original image.
16 Cameraman simulations I 8000 Magnitude Row Index Column index (a) Cameraman in space domain (b) Cameraman in DCT domain Figure : The original cameraman image (resized to pixels) in space domain and DCT domain.
17 Cameraman simulations II PSNR (db) GTCS S GTCS P KCS Reconstruction time (sec) GTCS S GTCS P KCS Normalized number of samples (K*K/N) (a) PSNR comparison Normalized number of samples (K*K/N) (b) Recovery time comparison Figure : PSNR and reconstruction time comparison on compressible image.
18 Cameraman simulations III (a) GTCS-S, K = 46, PSNR = db (b) GTCS-P/MWCS, K = 46, PSNR = db (c) KCS, K = 46, PSNR = db (d) GTCS-S,K = 63, PSNR = db (e) GTCS-P/MWCS, K = 63, PSNR = db (f) KCS, K = 63, PSNR = db
19 Cameraman explanations As shown in Figure??, the cameraman image is resized to (N = 4096 pixels). The image itself is non-sparse. However, in some transformed domain, such as discrete cosine transformation (DCT) domain in this case, the magnitudes of the coefficients decay by power law in both directions (see Figure??), thus are compressible. We let the number of measurements evenly split among the two modes. Again, in matrix data case, MWCS concurs with GTCS-P. We exhaustively vary K from 1 to 64. Figure?? and?? compare the PSNR and the recovery time respectively. Unlike the sparse image case, GTCS-P shows outstanding performance in comparison with all other methods, in terms of both accuracy and speed, followed by KCS and then GTCS-S. The reconstructed images when K = 46, using 0.51 normalized number of samples and when K = 63, using 0.96 normalized number of samples are shown in Figure??.
20 Compressive sensing of tensors M = (M 1,..., M d ), N = (N 1,..., N d ) N d, J = {j 1,..., j k } [d] Tensors: d i=1 RN i = R N 1... N d = R N Contraction of A = [a ij1,...,i jk ] jp JR N jp with T = [t i1,...,i d ] R N : A T = i j p [N jp ],jp J a i j1,...,i jk t i1,...,i d l [d]\j R N l X = [x i1,...,i d ] R N, U = U 1 U 2... U d R (M 1,N 1,M 2,N 2,...,M d,n d ) U p = [u (p) i pj p ] R Mp Np, p [d], U Kronecker product of U 1,..., U d. Y = [y i1,...,i d ] = X U := X 1 U 1 2 U 2... d U d R M y i1,...,i p = j q [N q],q [d] x j 1,...,j d q [d] u i q,j q Thm X is s-sparse, each U i has s-null property then X uniquely recovered from Y. Algo 1: GTCS-S Algo 2: GTCS-P
21 Algo 1- GTCS-S Unfold Y in mode 1: Y (1) = U 1 W 1 R M 1 (M 2... M d ), W 1 := X (1) [ 2 k=d U k] R N 1 (M 2... M d ) As for matrices recover the M 2 := M 2 M d columns of W 1 using U 1 Complexity: O( M 2 N1 3). Now we need to recover Y 1 := X 1 I 1 2 U 2... d U d RN 1 M 2... M d Equivalently, recover N 1, d 1 mode tensors in R N 2... N d R M 2... M d using d 1 matrices U 2,..., U d. Complexity d i=1 Ñi 1 M i+1 N 3 i from Ñ 0 = M d+1 = 1, Ñ i = N 1... N i, Mi = M i... M d d = 3: M 2 M 3 N N 1M 3 N N 1N 2 N 3 3
22 Algo 2- GTCS-P Unfold X in mode k: X (k) R N k N N k, N = d i=1 N i. As X is s-sparse rank k X := rank X (k) s. Y (k) = U k X (k) [ i k U i ] Range Y (k) U k Range X (k), rank Y (k) s. X (1) = R 1 j=1 u iv i, u 1,..., u R1 spans range of X (1) so R 1 s Each v i corresponds to U i R N 2...N d So (1) X = R j=1 u 1,j... u d,j, R s d 1 u k,1,..., u k,r R N k which is s-sparse span Range X (k) and each is s-sparse Compute decomposition Y = R j=1 w 1,j... w d,j, R s d 1, w k,1,..., w k,r R M k span Range Y (k), Compl: O(s d 1 d i=1 M i) Find u k,j from w k,j = U k u k,j and reconstruct X from (1) Complexity O(ds d 1 max(n 1,..., N d ) 3 ), s = O(log(max(N 1,..., N d )))
23 Summary of complexity converting linear data N i = N α i, M i = O(log N), α i > 0, d i=1 α i = 1, s = log N d = 3 GTCS-S: O((log N) 2 N ) GTCS-P: O((log N) 2 N) GTCS-P: O((log N) d 1 N 3 d ) for any d. Warning: the roundoff error in computing parfac decomposition of Y and then of X increases significantly with d.
24 Sparse video representation We compare the performance of GTCS and KCS on video data. Each frame of the video sequence is preprocessed to have size and we choose the first 24 frames. The video data together is represented by a tensor and has N = voxels in total. To obtain a sparse tensor, we manually keep only nonzero entries in the center of the video tensor data and the rest are set to zero. The video tensor is 216-sparse and its mode-i fibers are all 6-sparse i = 1, 2, 3. The randomly constructed Gaussian measurement matrix for each mode is now of size K 24 and the total number of samples is K 3. The normalized number of samples is K 3 N. We vary K from 1 to 13.
25 PSNR and reconstruction time of sparse video PSNR (db) GTCS S GTCS P KCS Reconstruction time (sec) GTCS S GTCS P KCS Normalized number of samples (K*K*K/N) (a) PSNR comparison Normalized number of samples (K*K*K/N) (b) Recovery time comparison Figure : PSNR and reconstruction time comparison on sparse video.
26 Reconstruction errors of sparse video Reconstruction error x Row index 20 5 x Column index Reconstruction error Row index Column index Reconstruction error Row index Column index x (a) Reconstruction error of GTCS-S (b) Reconstruction error of GTCS-P (c) Reconstruction error of KCS Figure : Visualization of the reconstruction error in the recovered video frame 9 by GTCS-S (PSNR = db), GTCS-P (PSNR = db) and KCS (PSNR = db) when K = 12, using normalized number of samples.
27 Conclusion Real-world signals as color imaging, video sequences and multi-sensor networks, are generated by the interaction of multiple factors or multimedia and can be represented by higher-order tensors. We propose Generalized Tensor Compressive Sensing (GTCS)-a unified framework for compressive sensing of sparse higher-order tensors. We give two reconstruction procedures, a serial method (GTCS-S) and a parallelizable method (GTCS-P). We compare the performance of GTCS with KCS and MWCS experimentally on various types of data including sparse image, compressible image, sparse video and compressible video. Experimental results show that GTCS outperforms KCS and MWCS in terms of both accuracy and efficiency. Compared to KCS, our recovery problems are in terms of each tensor mode, which is much smaller comparing with the vectorization of all tensor modes. Unlike MWCS, GTCS manages to get rid of tensor rank estimation, which considerably reduces the computational complexity and at the same time improves the reconstruction accuracy.
28 References 1 C. Caiafa and A. Cichocki, Multidimensional compressed sensing and their applications, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(6), , (2013). E. Candes, J. Romberg and T. Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete information, Information Theory, IEEE Transactions on 52 (2006), E. Candes and T. Tao, Near optimal signal recovery from random projections: Universal encoding strategies, Information Theory, IEEE Transactions on 52 (2006), D. Donoho, Compressed sensing, Information Theory, IEEE Transactions on 52 (2006), M. Duarte and R. Baraniuk, Kronecker compressive sensing, Image Processing, IEEE Transactions on, 2 (2012),
29 References 2 Q. Li, D. Schonfeld and S. Friedland, Generalized tensor compressive sensing, Multimedia and Expo (ICME), 2013 IEEE International Conference on, July 2013, ISSN: , 6 pages. S. Friedland, Q. Li and D. Schonfeld, Compressive Sensing of Sparse Tensors, arxiv: N. Sidiropoulus and A. Kyrillidis, Multi-way compressive sensing for sparse low-rank tensors, Signal Processing Letters, IEEE, 19 (2012),
Computational Problems in Tensors
Shmuel Friedland Univ. Illinois at Chicago linear Algebra ICERM June 5, 2014 Overview Uniqueness of best approximation Primer on tensors Best rank one approximation of tensors Number of critical points
More informationCompressive Sensing of Sparse Tensors
MANUSCRIPT OF IEEE TRANSACTIONS ON IMAGE PROCESSING, EDICS: SMR-REP Compressive Sensing of Sparse Tensors Shmuel Friedland, Qun Li*, Member, IEEE, and Dan Schonfeld, Fellow, IEEE Abstract Compressive sensing
More informationTruncation Strategy of Tensor Compressive Sensing for Noisy Video Sequences
Journal of Information Hiding and Multimedia Signal Processing c 2016 ISSN 207-4212 Ubiquitous International Volume 7, Number 5, September 2016 Truncation Strategy of Tensor Compressive Sensing for Noisy
More informationStrengthened Sobolev inequalities for a random subspace of functions
Strengthened Sobolev inequalities for a random subspace of functions Rachel Ward University of Texas at Austin April 2013 2 Discrete Sobolev inequalities Proposition (Sobolev inequality for discrete images)
More informationRandomness-in-Structured Ensembles for Compressed Sensing of Images
Randomness-in-Structured Ensembles for Compressed Sensing of Images Abdolreza Abdolhosseini Moghadam Dep. of Electrical and Computer Engineering Michigan State University Email: abdolhos@msu.edu Hayder
More informationCompressed sensing techniques for hyperspectral image recovery
Compressed sensing techniques for hyperspectral image recovery A. Abrardo, M. Barni, C. M. Carretti, E. Magli, S. Kuiteing Kamdem, R. Vitulli ABSTRACT Compressed Sensing (CS) theory is progressively gaining
More informationIntroduction to Compressed Sensing
Introduction to Compressed Sensing Alejandro Parada, Gonzalo Arce University of Delaware August 25, 2016 Motivation: Classical Sampling 1 Motivation: Classical Sampling Issues Some applications Radar Spectral
More informationCompressive Sensing with Random Matrices
Compressive Sensing with Random Matrices Lucas Connell University of Georgia 9 November 017 Lucas Connell (University of Georgia) Compressive Sensing with Random Matrices 9 November 017 1 / 18 Overview
More informationSparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation
Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation Alfredo Nava-Tudela John J. Benedetto, advisor 5/10/11 AMSC 663/664 1 Problem Let A be an n
More informationIntroduction How it works Theory behind Compressed Sensing. Compressed Sensing. Huichao Xue. CS3750 Fall 2011
Compressed Sensing Huichao Xue CS3750 Fall 2011 Table of Contents Introduction From News Reports Abstract Definition How it works A review of L 1 norm The Algorithm Backgrounds for underdetermined linear
More informationCOMPRESSED SENSING IN PYTHON
COMPRESSED SENSING IN PYTHON Sercan Yıldız syildiz@samsi.info February 27, 2017 OUTLINE A BRIEF INTRODUCTION TO COMPRESSED SENSING A BRIEF INTRODUCTION TO CVXOPT EXAMPLES A Brief Introduction to Compressed
More informationNear Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing
Near Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas M.Vidyasagar@utdallas.edu www.utdallas.edu/ m.vidyasagar
More informationCompressed Sensing and Robust Recovery of Low Rank Matrices
Compressed Sensing and Robust Recovery of Low Rank Matrices M. Fazel, E. Candès, B. Recht, P. Parrilo Electrical Engineering, University of Washington Applied and Computational Mathematics Dept., Caltech
More informationarxiv: v1 [cs.it] 26 Oct 2018
Outlier Detection using Generative Models with Theoretical Performance Guarantees arxiv:1810.11335v1 [cs.it] 6 Oct 018 Jirong Yi Anh Duc Le Tianming Wang Xiaodong Wu Weiyu Xu October 9, 018 Abstract This
More informationAN INTRODUCTION TO COMPRESSIVE SENSING
AN INTRODUCTION TO COMPRESSIVE SENSING Rodrigo B. Platte School of Mathematical and Statistical Sciences APM/EEE598 Reverse Engineering of Complex Dynamical Networks OUTLINE 1 INTRODUCTION 2 INCOHERENCE
More informationNew Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit
New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence
More informationInterpolation via weighted l 1 minimization
Interpolation via weighted l 1 minimization Rachel Ward University of Texas at Austin December 12, 2014 Joint work with Holger Rauhut (Aachen University) Function interpolation Given a function f : D C
More informationCompressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery
Compressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery Jorge F. Silva and Eduardo Pavez Department of Electrical Engineering Information and Decision Systems Group Universidad
More informationExact Low-rank Matrix Recovery via Nonconvex M p -Minimization
Exact Low-rank Matrix Recovery via Nonconvex M p -Minimization Lingchen Kong and Naihua Xiu Department of Applied Mathematics, Beijing Jiaotong University, Beijing, 100044, People s Republic of China E-mail:
More informationSignal Recovery, Uncertainty Relations, and Minkowski Dimension
Signal Recovery, Uncertainty Relations, and Minkowski Dimension Helmut Bőlcskei ETH Zurich December 2013 Joint work with C. Aubel, P. Kuppinger, G. Pope, E. Riegler, D. Stotz, and C. Studer Aim of this
More informationConstructing Explicit RIP Matrices and the Square-Root Bottleneck
Constructing Explicit RIP Matrices and the Square-Root Bottleneck Ryan Cinoman July 18, 2018 Ryan Cinoman Constructing Explicit RIP Matrices July 18, 2018 1 / 36 Outline 1 Introduction 2 Restricted Isometry
More informationA new method on deterministic construction of the measurement matrix in compressed sensing
A new method on deterministic construction of the measurement matrix in compressed sensing Qun Mo 1 arxiv:1503.01250v1 [cs.it] 4 Mar 2015 Abstract Construction on the measurement matrix A is a central
More informationLarge-Scale L1-Related Minimization in Compressive Sensing and Beyond
Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Arizona State University March
More informationRecovering overcomplete sparse representations from structured sensing
Recovering overcomplete sparse representations from structured sensing Deanna Needell Claremont McKenna College Feb. 2015 Support: Alfred P. Sloan Foundation and NSF CAREER #1348721. Joint work with Felix
More informationCSC 576: Variants of Sparse Learning
CSC 576: Variants of Sparse Learning Ji Liu Department of Computer Science, University of Rochester October 27, 205 Introduction Our previous note basically suggests using l norm to enforce sparsity in
More informationMultipath Matching Pursuit
Multipath Matching Pursuit Submitted to IEEE trans. on Information theory Authors: S. Kwon, J. Wang, and B. Shim Presenter: Hwanchol Jang Multipath is investigated rather than a single path for a greedy
More informationSolution Recovery via L1 minimization: What are possible and Why?
Solution Recovery via L1 minimization: What are possible and Why? Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Eighth US-Mexico Workshop on Optimization
More informationExponential decay of reconstruction error from binary measurements of sparse signals
Exponential decay of reconstruction error from binary measurements of sparse signals Deanna Needell Joint work with R. Baraniuk, S. Foucart, Y. Plan, and M. Wootters Outline Introduction Mathematical Formulation
More informationSparse and Low Rank Recovery via Null Space Properties
Sparse and Low Rank Recovery via Null Space Properties Holger Rauhut Lehrstuhl C für Mathematik (Analysis), RWTH Aachen Convexity, probability and discrete structures, a geometric viewpoint Marne-la-Vallée,
More informationModel-Based Compressive Sensing for Signal Ensembles. Marco F. Duarte Volkan Cevher Richard G. Baraniuk
Model-Based Compressive Sensing for Signal Ensembles Marco F. Duarte Volkan Cevher Richard G. Baraniuk Concise Signal Structure Sparse signal: only K out of N coordinates nonzero model: union of K-dimensional
More informationCompressed Sensing and Related Learning Problems
Compressed Sensing and Related Learning Problems Yingzhen Li Dept. of Mathematics, Sun Yat-sen University Advisor: Prof. Haizhang Zhang Advisor: Prof. Haizhang Zhang 1 / Overview Overview Background Compressed
More informationCompressed Sensing and Sparse Recovery
ELE 538B: Sparsity, Structure and Inference Compressed Sensing and Sparse Recovery Yuxin Chen Princeton University, Spring 217 Outline Restricted isometry property (RIP) A RIPless theory Compressed sensing
More informationKronecker Compressive Sensing
Kronecker Compressive Sensing Marco F. Duarte, Member, IEEE, and Richard G. Baraniuk, Fellow, IEEE Abstract Compressive sensing (CS) is an emerging approach for the acquisition of signals having a sparse
More informationGREEDY SIGNAL RECOVERY REVIEW
GREEDY SIGNAL RECOVERY REVIEW DEANNA NEEDELL, JOEL A. TROPP, ROMAN VERSHYNIN Abstract. The two major approaches to sparse recovery are L 1-minimization and greedy methods. Recently, Needell and Vershynin
More informationCS 229r: Algorithms for Big Data Fall Lecture 19 Nov 5
CS 229r: Algorithms for Big Data Fall 215 Prof. Jelani Nelson Lecture 19 Nov 5 Scribe: Abdul Wasay 1 Overview In the last lecture, we started discussing the problem of compressed sensing where we are given
More informationSPARSE signal representations have gained popularity in recent
6958 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 10, OCTOBER 2011 Blind Compressed Sensing Sivan Gleichman and Yonina C. Eldar, Senior Member, IEEE Abstract The fundamental principle underlying
More informationDoes Compressed Sensing have applications in Robust Statistics?
Does Compressed Sensing have applications in Robust Statistics? Salvador Flores December 1, 2014 Abstract The connections between robust linear regression and sparse reconstruction are brought to light.
More informationOptimization for Compressed Sensing
Optimization for Compressed Sensing Robert J. Vanderbei 2014 March 21 Dept. of Industrial & Systems Engineering University of Florida http://www.princeton.edu/ rvdb Lasso Regression The problem is to solve
More informationCombining Sparsity with Physically-Meaningful Constraints in Sparse Parameter Estimation
UIUC CSL Mar. 24 Combining Sparsity with Physically-Meaningful Constraints in Sparse Parameter Estimation Yuejie Chi Department of ECE and BMI Ohio State University Joint work with Yuxin Chen (Stanford).
More informationFast Hard Thresholding with Nesterov s Gradient Method
Fast Hard Thresholding with Nesterov s Gradient Method Volkan Cevher Idiap Research Institute Ecole Polytechnique Federale de ausanne volkan.cevher@epfl.ch Sina Jafarpour Department of Computer Science
More informationSelf-Calibration and Biconvex Compressive Sensing
Self-Calibration and Biconvex Compressive Sensing Shuyang Ling Department of Mathematics, UC Davis July 12, 2017 Shuyang Ling (UC Davis) SIAM Annual Meeting, 2017, Pittsburgh July 12, 2017 1 / 22 Acknowledgements
More informationSignal Recovery from Permuted Observations
EE381V Course Project Signal Recovery from Permuted Observations 1 Problem Shanshan Wu (sw33323) May 8th, 2015 We start with the following problem: let s R n be an unknown n-dimensional real-valued signal,
More informationDeterminis)c Compressed Sensing for Images using Chirps and Reed- Muller Sequences
Determinis)c Compressed Sensing for Images using Chirps and Reed- Muller Sequences Kangyu Ni Mathematics Arizona State University Joint with Somantika Datta*, Prasun Mahanti, Svetlana Roudenko, Douglas
More informationEUSIPCO
EUSIPCO 013 1569746769 SUBSET PURSUIT FOR ANALYSIS DICTIONARY LEARNING Ye Zhang 1,, Haolong Wang 1, Tenglong Yu 1, Wenwu Wang 1 Department of Electronic and Information Engineering, Nanchang University,
More informationThe number of singular vector tuples and approximation of symmetric tensors / 13
The number of singular vector tuples and approximation of symmetric tensors Shmuel Friedland Univ. Illinois at Chicago Colloquium NYU Courant May 12, 2014 Joint results with Giorgio Ottaviani and Margaret
More informationPre-weighted Matching Pursuit Algorithms for Sparse Recovery
Journal of Information & Computational Science 11:9 (214) 2933 2939 June 1, 214 Available at http://www.joics.com Pre-weighted Matching Pursuit Algorithms for Sparse Recovery Jingfei He, Guiling Sun, Jie
More informationCS on CS: Computer Science insights into Compresive Sensing (and vice versa) Piotr Indyk MIT
CS on CS: Computer Science insights into Compresive Sensing (and vice versa) Piotr Indyk MIT Sparse Approximations Goal: approximate a highdimensional vector x by x that is sparse, i.e., has few nonzero
More informationIEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER
IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER 2015 1239 Preconditioning for Underdetermined Linear Systems with Sparse Solutions Evaggelia Tsiligianni, StudentMember,IEEE, Lisimachos P. Kondi,
More informationCombining geometry and combinatorics
Combining geometry and combinatorics A unified approach to sparse signal recovery Anna C. Gilbert University of Michigan joint work with R. Berinde (MIT), P. Indyk (MIT), H. Karloff (AT&T), M. Strauss
More informationLecture: Introduction to Compressed Sensing Sparse Recovery Guarantees
Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html Acknowledgement: this slides is based on Prof. Emmanuel Candes and Prof. Wotao Yin
More informationSparse Solutions of an Undetermined Linear System
1 Sparse Solutions of an Undetermined Linear System Maddullah Almerdasy New York University Tandon School of Engineering arxiv:1702.07096v1 [math.oc] 23 Feb 2017 Abstract This work proposes a research
More informationCompressed Sensing: Lecture I. Ronald DeVore
Compressed Sensing: Lecture I Ronald DeVore Motivation Compressed Sensing is a new paradigm for signal/image/function acquisition Motivation Compressed Sensing is a new paradigm for signal/image/function
More informationCOMPRESSIVE SAMPLING USING EM ALGORITHM. Technical Report No: ASU/2014/4
COMPRESSIVE SAMPLING USING EM ALGORITHM ATANU KUMAR GHOSH, ARNAB CHAKRABORTY Technical Report No: ASU/2014/4 Date: 29 th April, 2014 Applied Statistics Unit Indian Statistical Institute Kolkata- 700018
More informationCompressed sensing. Or: the equation Ax = b, revisited. Terence Tao. Mahler Lecture Series. University of California, Los Angeles
Or: the equation Ax = b, revisited University of California, Los Angeles Mahler Lecture Series Acquiring signals Many types of real-world signals (e.g. sound, images, video) can be viewed as an n-dimensional
More informationA Generalized Restricted Isometry Property
1 A Generalized Restricted Isometry Property Jarvis Haupt and Robert Nowak Department of Electrical and Computer Engineering, University of Wisconsin Madison University of Wisconsin Technical Report ECE-07-1
More informationFast and Robust Phase Retrieval
Fast and Robust Phase Retrieval Aditya Viswanathan aditya@math.msu.edu CCAM Lunch Seminar Purdue University April 18 2014 0 / 27 Joint work with Yang Wang Mark Iwen Research supported in part by National
More informationECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis Lecture 7: Matrix completion Yuejie Chi The Ohio State University Page 1 Reference Guaranteed Minimum-Rank Solutions of Linear
More informationCompressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
More informationDeterministic sampling masks and compressed sensing: Compensating for partial image loss at the pixel level
Deterministic sampling masks and compressed sensing: Compensating for partial image loss at the pixel level Alfredo Nava-Tudela Institute for Physical Science and Technology and Norbert Wiener Center,
More informationElaine T. Hale, Wotao Yin, Yin Zhang
, Wotao Yin, Yin Zhang Department of Computational and Applied Mathematics Rice University McMaster University, ICCOPT II-MOPTA 2007 August 13, 2007 1 with Noise 2 3 4 1 with Noise 2 3 4 1 with Noise 2
More informationExact Topology Identification of Large-Scale Interconnected Dynamical Systems from Compressive Observations
Exact Topology Identification of arge-scale Interconnected Dynamical Systems from Compressive Observations Borhan M Sanandaji, Tyrone Vincent, and Michael B Wakin Abstract In this paper, we consider the
More informationarxiv: v1 [cs.it] 21 Feb 2013
q-ary Compressive Sensing arxiv:30.568v [cs.it] Feb 03 Youssef Mroueh,, Lorenzo Rosasco, CBCL, CSAIL, Massachusetts Institute of Technology LCSL, Istituto Italiano di Tecnologia and IIT@MIT lab, Istituto
More informationAnalog-to-Information Conversion
Analog-to-Information Conversion Sergiy A. Vorobyov Dept. Signal Processing and Acoustics, Aalto University February 2013 Winter School on Compressed Sensing, Ruka 1/55 Outline 1 Compressed Sampling (CS)
More informationLinear Sketches A Useful Tool in Streaming and Compressive Sensing
Linear Sketches A Useful Tool in Streaming and Compressive Sensing Qin Zhang 1-1 Linear sketch Random linear projection M : R n R k that preserves properties of any v R n with high prob. where k n. M =
More informationLecture 22: More On Compressed Sensing
Lecture 22: More On Compressed Sensing Scribed by Eric Lee, Chengrun Yang, and Sebastian Ament Nov. 2, 207 Recap and Introduction Basis pursuit was the method of recovering the sparsest solution to an
More informationThresholds for the Recovery of Sparse Solutions via L1 Minimization
Thresholds for the Recovery of Sparse Solutions via L Minimization David L. Donoho Department of Statistics Stanford University 39 Serra Mall, Sequoia Hall Stanford, CA 9435-465 Email: donoho@stanford.edu
More informationLIMITATION OF LEARNING RANKINGS FROM PARTIAL INFORMATION. By Srikanth Jagabathula Devavrat Shah
00 AIM Workshop on Ranking LIMITATION OF LEARNING RANKINGS FROM PARTIAL INFORMATION By Srikanth Jagabathula Devavrat Shah Interest is in recovering distribution over the space of permutations over n elements
More informationKronecker Product Matrices for Compressive Sensing
Kronecker Product Matrices for Compressive Sensing Marco F. Duarte and Richard G. Baraniuk Technical Report TREE-1105 Department of Electrical and Computer Engineering Rice University March 17, 2011 Abstract
More informationReconstruction from Anisotropic Random Measurements
Reconstruction from Anisotropic Random Measurements Mark Rudelson and Shuheng Zhou The University of Michigan, Ann Arbor Coding, Complexity, and Sparsity Workshop, 013 Ann Arbor, Michigan August 7, 013
More informationNoisy Signal Recovery via Iterative Reweighted L1-Minimization
Noisy Signal Recovery via Iterative Reweighted L1-Minimization Deanna Needell UC Davis / Stanford University Asilomar SSC, November 2009 Problem Background Setup 1 Suppose x is an unknown signal in R d.
More informationINDUSTRIAL MATHEMATICS INSTITUTE. B.S. Kashin and V.N. Temlyakov. IMI Preprint Series. Department of Mathematics University of South Carolina
INDUSTRIAL MATHEMATICS INSTITUTE 2007:08 A remark on compressed sensing B.S. Kashin and V.N. Temlyakov IMI Preprint Series Department of Mathematics University of South Carolina A remark on compressed
More informationThe Secrecy of Compressed Sensing Measurements
The Secrecy of Compressed Sensing Measurements Yaron Rachlin and Dror Baron Abstract Results in compressed sensing describe the feasibility of reconstructing sparse signals using a small number of linear
More informationCollaborative Compressive Spectrum Sensing Using Kronecker Sparsifying Basis
Collaborative Compressive Spectrum Sensing Using Kronecker Sparsifying Basis Ahmed M Elzanati, Mohamed F Abdelkader, Karim G Seddik and Atef M Ghuniem Department of Communication and Electronics, Sinai
More informationA Structured Construction of Optimal Measurement Matrix for Noiseless Compressed Sensing via Polarization of Analog Transmission
Li and Kang: A Structured Construction of Optimal Measurement Matrix for Noiseless Compressed Sensing 1 A Structured Construction of Optimal Measurement Matrix for Noiseless Compressed Sensing via Polarization
More informationA Power Efficient Sensing/Communication Scheme: Joint Source-Channel-Network Coding by Using Compressive Sensing
Forty-Ninth Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 28-30, 20 A Power Efficient Sensing/Communication Scheme: Joint Source-Channel-Network Coding by Using Compressive Sensing
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationAn Overview of Compressed Sensing
An Overview of Compressed Sensing Nathan Schneider November 18, 2009 Abstract In a large number of applications, the system will be designed to sample at a rate equal to at least the frequency bandwidth
More informationRecent Developments in Compressed Sensing
Recent Developments in Compressed Sensing M. Vidyasagar Distinguished Professor, IIT Hyderabad m.vidyasagar@iith.ac.in, www.iith.ac.in/ m vidyasagar/ ISL Seminar, Stanford University, 19 April 2018 Outline
More informationSolving Underdetermined Linear Equations and Overdetermined Quadratic Equations (using Convex Programming)
Solving Underdetermined Linear Equations and Overdetermined Quadratic Equations (using Convex Programming) Justin Romberg Georgia Tech, ECE Caltech ROM-GR Workshop June 7, 2013 Pasadena, California Linear
More informationof Orthogonal Matching Pursuit
A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit Qun Mo arxiv:50.0708v [cs.it] 8 Jan 205 Abstract We shall show that if the restricted isometry constant (RIC) δ s+ (A) of the measurement
More informationApplication to Hyperspectral Imaging
Compressed Sensing of Low Complexity High Dimensional Data Application to Hyperspectral Imaging Kévin Degraux PhD Student, ICTEAM institute Université catholique de Louvain, Belgium 6 November, 2013 Hyperspectral
More informationSparse recovery for spherical harmonic expansions
Rachel Ward 1 1 Courant Institute, New York University Workshop Sparsity and Cosmology, Nice May 31, 2011 Cosmic Microwave Background Radiation (CMB) map Temperature is measured as T (θ, ϕ) = k k=0 l=
More informationStochastic geometry and random matrix theory in CS
Stochastic geometry and random matrix theory in CS IPAM: numerical methods for continuous optimization University of Edinburgh Joint with Bah, Blanchard, Cartis, and Donoho Encoder Decoder pair - Encoder/Decoder
More informationLeast-Squares Performance of Analog Product Codes
Copyright 004 IEEE Published in the Proceedings of the Asilomar Conference on Signals, Systems and Computers, 7-0 ovember 004, Pacific Grove, California, USA Least-Squares Performance of Analog Product
More informationEnhanced Compressive Sensing and More
Enhanced Compressive Sensing and More Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Nonlinear Approximation Techniques Using L1 Texas A & M University
More informationSparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images
Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images Alfredo Nava-Tudela ant@umd.edu John J. Benedetto Department of Mathematics jjb@umd.edu Abstract In this project we are
More informationSparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery
Sparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery Anna C. Gilbert Department of Mathematics University of Michigan Sparse signal recovery measurements:
More informationA New Estimate of Restricted Isometry Constants for Sparse Solutions
A New Estimate of Restricted Isometry Constants for Sparse Solutions Ming-Jun Lai and Louis Y. Liu January 12, 211 Abstract We show that as long as the restricted isometry constant δ 2k < 1/2, there exist
More informationCompressed Sensing Using Reed- Solomon and Q-Ary LDPC Codes
Compressed Sensing Using Reed- Solomon and Q-Ary LDPC Codes Item Type text; Proceedings Authors Jagiello, Kristin M. Publisher International Foundation for Telemetering Journal International Telemetering
More informationCompressive Sensing Theory and L1-Related Optimization Algorithms
Compressive Sensing Theory and L1-Related Optimization Algorithms Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, USA CAAM Colloquium January 26, 2009 Outline:
More informationLow-Rank Tensor Completion by Truncated Nuclear Norm Regularization
Low-Rank Tensor Completion by Truncated Nuclear Norm Regularization Shengke Xue, Wenyuan Qiu, Fan Liu, and Xinyu Jin College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou,
More informationRecovery Based on Kolmogorov Complexity in Underdetermined Systems of Linear Equations
Recovery Based on Kolmogorov Complexity in Underdetermined Systems of Linear Equations David Donoho Department of Statistics Stanford University Email: donoho@stanfordedu Hossein Kakavand, James Mammen
More informationSparse Optimization Lecture: Sparse Recovery Guarantees
Those who complete this lecture will know Sparse Optimization Lecture: Sparse Recovery Guarantees Sparse Optimization Lecture: Sparse Recovery Guarantees Instructor: Wotao Yin Department of Mathematics,
More informationCompressed Sensing. 1 Introduction. 2 Design of Measurement Matrices
Compressed Sensing Yonina C. Eldar Electrical Engineering Department, Technion-Israel Institute of Technology, Haifa, Israel, 32000 1 Introduction Compressed sensing (CS) is an exciting, rapidly growing
More informationGreedy Signal Recovery and Uniform Uncertainty Principles
Greedy Signal Recovery and Uniform Uncertainty Principles SPIE - IE 2008 Deanna Needell Joint work with Roman Vershynin UC Davis, January 2008 Greedy Signal Recovery and Uniform Uncertainty Principles
More informationCompressed Sensing and Neural Networks
and Jan Vybíral (Charles University & Czech Technical University Prague, Czech Republic) NOMAD Summer Berlin, September 25-29, 2017 1 / 31 Outline Lasso & Introduction Notation Training the network Applications
More informationSparse Legendre expansions via l 1 minimization
Sparse Legendre expansions via l 1 minimization Rachel Ward, Courant Institute, NYU Joint work with Holger Rauhut, Hausdorff Center for Mathematics, Bonn, Germany. June 8, 2010 Outline Sparse recovery
More informationRui ZHANG Song LI. Department of Mathematics, Zhejiang University, Hangzhou , P. R. China
Acta Mathematica Sinica, English Series May, 015, Vol. 31, No. 5, pp. 755 766 Published online: April 15, 015 DOI: 10.1007/s10114-015-434-4 Http://www.ActaMath.com Acta Mathematica Sinica, English Series
More informationOn Optimal Frame Conditioners
On Optimal Frame Conditioners Chae A. Clark Department of Mathematics University of Maryland, College Park Email: cclark18@math.umd.edu Kasso A. Okoudjou Department of Mathematics University of Maryland,
More informationLecture Notes 9: Constrained Optimization
Optimization-based data analysis Fall 017 Lecture Notes 9: Constrained Optimization 1 Compressed sensing 1.1 Underdetermined linear inverse problems Linear inverse problems model measurements of the form
More information