Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery
|
|
- Morris McDowell
- 6 years ago
- Views:
Transcription
1 Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan
2 Connection between... Sparse Approximation and Compressed Sensing
3 Encoding schemes Image Linear encoding Φ Matrix image independent Nonlinear decoding Approx. Image Nonlinear encoding Ω Linear decoding Matrix image dependent Approx. Image Sparse Approximation Compressed Sensing
4 Linear encoding, nonlinear decoding Φ x = c highly nonlinear decoding x Φ y = d highly nonlinear decoding y Measure accuracy of decoded signal with respect to x k, best k-term approximation for x (in some orthonormal basis)
5 Problem statement m as small as possible (Design) matrix Φ: R n R m (Design Φ so that) given Φx = y for any signal x R n, there is an algorithm to recover x with x x p C x x k q
6 Parameters Number of measurements m Recovery time Approximation guarantee (p, q norms, mixed) One matrix vs. distribution over matrices Explicit construction Tolerance to measurement noise
7 Comparison with Sparse Approximation Sparse: Given y and Φ, find (sparse) x such that y = Φx. Return x with guarantee Φ x y 2 small compared with y Φx k 2. CS: Given y and Φ, find (sparse) x such that y = Φx. Return x with guarantee x x 2 small compared with 1 k x x k 1.
8 Analogy: root-finding p with p p ɛ p with f( p) 0 ɛ root = p ɛ ɛ Sparse: Given f (and y = 0), find p such that f (p) = 0. Return p with guarantee f ( p) 0 small. CS: Given f (and y = 0), find p such that f (p) = 0. Return p with guarantee p p small.
9 Root-finding analogy? Φx = x x !0.5 2!1 1.5!1.5 1!2!2!1.5!1!
10 Algorithms for CS Convex optimization Greedy iterative methods
11 Algorithms for CS Convex optimization Greedy iterative methods Problem formulations Recover entire signal Recover k significant terms
12 Algorithms for CS Convex optimization Greedy iterative methods Problem formulations Recover entire signal Recover k significant terms Role of probability: Probabilistic method: if choose Φ from distribution, whp it will satisfy certain properties Use randomness in algorithm
13 Algorithms for CS Convex optimization Greedy iterative methods Problem formulations Recover entire signal Recover k significant terms Role of probability: Probabilistic method: if choose Φ from distribution, whp it will satisfy certain properties Use randomness in algorithm Sparse representation is central to successful algorithms
14 CS Geometric methods Suppose Φ satisfies RIP(2, 2k, δ k ) condition: for any 2k-sparse vector x, (1 δ k ) x 2 Φx 2 (1 + δ k ) x 2. Given Φx = y, the solution x to the convex relaxation problem x = argmin z 1 s.t. Φz = Φx satisfies x x 2 C/ k x x k 1.
15 CS Geometric methods Suppose Φ satisfies RIP(2, 2k, δ k ) condition: for any 2k-sparse vector x, (1 δ k ) x 2 Φx 2 (1 + δ k ) x 2. Given Φx = y, the solution x to the convex relaxation problem x = argmin z 1 s.t. Φz = Φx satisfies x x 2 C/ k x x k 1. Dense recovery matrices: if draw Φ from random matrix with iid (sub-)gaussian entries or random rows of Fourier matrix, m = O(k log n/k) rows and n columns, Φ satisfies RIP(2) with high probability Example of probabilistic method to generate matrix not constructive Uniform guarantee: one matrix Φ that works for all x [Donoho 04],[Candes-Tao 04, 06],[Candes-Romberg-Tao 05], [Rudelson-Vershynin 06], [Cohen-Dahmen-DeVore 06], and many others...
16 CS Greedy algorithms Suppose Φ satisfies RIP(2, 2k, δ) condition. Given Φx = y, there are greedy iterative algorithms that produce x with x 0 = k x x 2 C( x x k 2 + 1/ k x x k 1 ). [Tropp-Needell 07],[Blumensath-Davies 08], and others Architecture of algorithms is Greedy Pursuit OMP Maximize: choose λ = set of m columns of Φ with large dot product Φ T r Update: Λ = Λ λ a = linear combination over Φ Λ Iterate: r = y Φa
17 Computational costs Computational time All dominated by Φ T r, matrix-vector product LP: O(T n), greedy: O(nk log(k/n) log( x 2 )) Storage Have to store Φ k log(n/k) n matrix (unless it has special structure!) Store x as you build it (only non-zero entries or entire vector) Randomness Have to generate Φ Theoretically, all entries are iid, truly random Note: drand48 is pseudo-random
18 Connection between... Sparse Approximation/Compressive Sensing and Streaming/Sublinear Algorithms
19 Data streams Data Streams IP packets header: source/destination address, payload size, protocol, etc. payload: transmitted data IP packets: Backbone link bandwidth = 40 Gbits/sec Over 90% packets are no more than 500 Bytes Monitor 10 million packets/second header: source/destination address, payload size, protocol, etc. payload: transmitted data
20 Heavy hitters ( , 128) ( , 1024) ( , 64) # Bytes IP address
21 Linear measurements ( , 128) =
22 Linear measurements ( , 128) ( , 1024) =
23 Linear measurements ( , 128) ( , 1024) ( , 64) =
24 Streaming algorithms Data (2) Storage space Auxiliary memory (4) what s stored in memory should be composable (1) per-item processing time (3) Time to produce output Resource constraints for (1), (2), and (3) should be poly log(d)
25 Streaming/Sub-linear algorithms There are sub-linear algorithms for CS running time: O(k polylog n) measurements/storage: O(k log(n/k)) error guarantees: match or l 1 /l 1 or l 2 /l 2 (with different probabilistic constructions) quite different geometric restrictions on Φ use and exploit pseudo-randomness to reduce storage space and speed up algorithms all conditions are sufficient, none seem to be necessary [Gilbert-Guha-Indyk-Kotidis-Muthukrishnan-Strauss 02], [Charikar-Chen-FarachColton 02] [Cormode-Muthukrishnan 04], [Gilbert-Strauss-Tropp-Vershynin 06, 07],[Gilbert-Li-Porat-Strauss 10]
26 New algorithms, phase transitions, random models d 1d histogram V-OPT (2,1) bi-criteria sparsity k d/2 d/log(d) 1/3 d random subdictionaries L1 optimization k ⅓μ -1 sufficient conditions for OMP and L1 optimizaiton 1 1/ d 1/3 1/2 1-1/d coherence μ 1
27 Alternative problem formulations + algorithms Dictionary Φ = piecewise constants (in 1 dimension) Extremely high coherence µ 1 1/d
28 Alternative problem formulations + algorithms Dictionary Φ = piecewise constants (in 1 dimension) Extremely high coherence µ 1 1/d Linear combinations of piecewise constants L1 L2 R2 R1
29 Alternative problem formulations + algorithms Dictionary Φ = piecewise constants (in 1 dimension) Extremely high coherence µ 1 1/d Linear combinations of piecewise constants L1 L2 R2 R1 but, more natural to count buckets in histograms L1 R1, L2 R2, L3 R3 Use at most twice as many buckets as piecewise constants in sparse representation
30 V-OPT Theorem There is a dynamic programming algorithm which produces the k-bucket histogram H k that minimizes x H k 2. The algorithm runs in time O(kd 2 ). [Jagadish, et al 98].
31 V-OPT Theorem There is a dynamic programming algorithm which produces the k-bucket histogram H k that minimizes x H k 2. The algorithm runs in time O(kd 2 ). [Jagadish, et al 98]. Dynamic programming is a different algorithmic technique from both greedy iterative algorithms and convex optimization.
32 V-OPT OPT(j,k-1) + cost(j+1,d) Idea: within a bucket, mean of signal values is best approximation 1 j+1 d
33 V-OPT OPT(j,k-1) + cost(j+1,d) Idea: within a bucket, mean of signal values is best approximation Assume last bucket is on [j + 1, d] 1 j+1 d
34 V-OPT OPT(j,k-1) + cost(j+1,d) 1 j+1 d Idea: within a bucket, mean of signal values is best approximation Assume last bucket is on [j + 1, d] What can we say about the remaining k 1 buckets?
35 V-OPT OPT(j,k-1) + cost(j+1,d) 1 j+1 d Idea: within a bucket, mean of signal values is best approximation Assume last bucket is on [j + 1, d] What can we say about the remaining k 1 buckets? must be optimal for range [1, j] with k 1 buckets
36 V-OPT OPT(j,k-1) + cost(j+1,d) 1 j+1 d where opt[d, k] = min {opt[j, k 1] + cost[(j + 1), d]} 1 j<d opt[j, k] = minimum cost of representing the set of values on [1, j] by histogram with k buckets and cost[(j + 1), d] = l 2 error on [j + 1, d]
37 V-OPT[Jagadish, et al 98] Input. signal x, buckets k Output. k bucket histogram H k x for i=1 to d for j=1 to k for l=1 to i-1 (split pt of j-1 bucket hist. and last bucket) OPT[i, j] = min OPT[i, j], OPT[l,j-1] + cost[l+1,i] OPT k d
38 Images Signals There is a big difference between 1 and any higher dimension for histograms
39 Images Signals There is a big difference between 1 and any higher dimension for histograms Finding the optimal k-rectangle histogram in 2d is NP-hard
40 Images Signals There is a big difference between 1 and any higher dimension for histograms Finding the optimal k-rectangle histogram in 2d is NP-hard There is an efficient algorithm which achieves minimal error (for k rectangles) with at most 4k rectangles. [Muthukrishnan and Strauss, 2003]
41 Summary Sparse approximation spawned many research directions, activities, applications, in many fields New algorithms, new applications Many new veins to be mined in next lectures
An Introduction to Sparse Approximation
An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,
More informationCombining geometry and combinatorics
Combining geometry and combinatorics A unified approach to sparse signal recovery Anna C. Gilbert University of Michigan joint work with R. Berinde (MIT), P. Indyk (MIT), H. Karloff (AT&T), M. Strauss
More informationSparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery
Sparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery Anna C. Gilbert Department of Mathematics University of Michigan Sparse signal recovery measurements:
More informationSparse analysis Lecture III: Dictionary geometry and greedy algorithms
Sparse analysis Lecture III: Dictionary geometry and greedy algorithms Anna C. Gilbert Department of Mathematics University of Michigan Intuition from ONB Key step in algorithm: r, ϕ j = x c i ϕ i, ϕ j
More informationCS on CS: Computer Science insights into Compresive Sensing (and vice versa) Piotr Indyk MIT
CS on CS: Computer Science insights into Compresive Sensing (and vice versa) Piotr Indyk MIT Sparse Approximations Goal: approximate a highdimensional vector x by x that is sparse, i.e., has few nonzero
More informationGreedy Signal Recovery and Uniform Uncertainty Principles
Greedy Signal Recovery and Uniform Uncertainty Principles SPIE - IE 2008 Deanna Needell Joint work with Roman Vershynin UC Davis, January 2008 Greedy Signal Recovery and Uniform Uncertainty Principles
More informationGREEDY SIGNAL RECOVERY REVIEW
GREEDY SIGNAL RECOVERY REVIEW DEANNA NEEDELL, JOEL A. TROPP, ROMAN VERSHYNIN Abstract. The two major approaches to sparse recovery are L 1-minimization and greedy methods. Recently, Needell and Vershynin
More informationCoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles
CoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles SIAM Student Research Conference Deanna Needell Joint work with Roman Vershynin and Joel Tropp UC Davis, May 2008 CoSaMP: Greedy Signal
More informationTutorial: Sparse Signal Recovery
Tutorial: Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan (Sparse) Signal recovery problem signal or population length N k important Φ x = y measurements or tests:
More informationLinear Sketches A Useful Tool in Streaming and Compressive Sensing
Linear Sketches A Useful Tool in Streaming and Compressive Sensing Qin Zhang 1-1 Linear sketch Random linear projection M : R n R k that preserves properties of any v R n with high prob. where k n. M =
More informationCoSaMP. Iterative signal recovery from incomplete and inaccurate samples. Joel A. Tropp
CoSaMP Iterative signal recovery from incomplete and inaccurate samples Joel A. Tropp Applied & Computational Mathematics California Institute of Technology jtropp@acm.caltech.edu Joint with D. Needell
More informationModel-Based Compressive Sensing for Signal Ensembles. Marco F. Duarte Volkan Cevher Richard G. Baraniuk
Model-Based Compressive Sensing for Signal Ensembles Marco F. Duarte Volkan Cevher Richard G. Baraniuk Concise Signal Structure Sparse signal: only K out of N coordinates nonzero model: union of K-dimensional
More informationLarge-Scale L1-Related Minimization in Compressive Sensing and Beyond
Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Arizona State University March
More informationSimultaneous Sparsity
Simultaneous Sparsity Joel A. Tropp Anna C. Gilbert Martin J. Strauss {jtropp annacg martinjs}@umich.edu Department of Mathematics The University of Michigan 1 Simple Sparse Approximation Work in the d-dimensional,
More informationUniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit
Claremont Colleges Scholarship @ Claremont CMC Faculty Publications and Research CMC Faculty Scholarship 6-5-2008 Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit
More informationSensing systems limited by constraints: physical size, time, cost, energy
Rebecca Willett Sensing systems limited by constraints: physical size, time, cost, energy Reduce the number of measurements needed for reconstruction Higher accuracy data subject to constraints Original
More informationNew Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit
New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence
More informationSolution Recovery via L1 minimization: What are possible and Why?
Solution Recovery via L1 minimization: What are possible and Why? Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Eighth US-Mexico Workshop on Optimization
More informationUniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit
Uniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit arxiv:0707.4203v2 [math.na] 14 Aug 2007 Deanna Needell Department of Mathematics University of California,
More informationAn Adaptive Sublinear Time Block Sparse Fourier Transform
An Adaptive Sublinear Time Block Sparse Fourier Transform Volkan Cevher Michael Kapralov Jonathan Scarlett Amir Zandieh EPFL February 8th 217 Given x C N, compute the Discrete Fourier Transform (DFT) of
More informationSparse recovery using sparse random matrices
Sparse recovery using sparse random matrices Radu Berinde MIT texel@mit.edu Piotr Indyk MIT indyk@mit.edu April 26, 2008 Abstract We consider the approximate sparse recovery problem, where the goal is
More informationSignal Recovery from Permuted Observations
EE381V Course Project Signal Recovery from Permuted Observations 1 Problem Shanshan Wu (sw33323) May 8th, 2015 We start with the following problem: let s R n be an unknown n-dimensional real-valued signal,
More informationAlgorithms for sparse analysis Lecture I: Background on sparse approximation
Algorithms for sparse analysis Lecture I: Background on sparse approximation Anna C. Gilbert Department of Mathematics University of Michigan Tutorial on sparse approximations and algorithms Compress data
More informationCompressive Sensing and Beyond
Compressive Sensing and Beyond Sohail Bahmani Gerorgia Tech. Signal Processing Compressed Sensing Signal Models Classics: bandlimited The Sampling Theorem Any signal with bandwidth B can be recovered
More informationRecent Developments in Compressed Sensing
Recent Developments in Compressed Sensing M. Vidyasagar Distinguished Professor, IIT Hyderabad m.vidyasagar@iith.ac.in, www.iith.ac.in/ m vidyasagar/ ISL Seminar, Stanford University, 19 April 2018 Outline
More informationSample Optimal Fourier Sampling in Any Constant Dimension
1 / 26 Sample Optimal Fourier Sampling in Any Constant Dimension Piotr Indyk Michael Kapralov MIT IBM Watson October 21, 2014 Fourier Transform and Sparsity Discrete Fourier Transform Given x C n, compute
More informationExponential decay of reconstruction error from binary measurements of sparse signals
Exponential decay of reconstruction error from binary measurements of sparse signals Deanna Needell Joint work with R. Baraniuk, S. Foucart, Y. Plan, and M. Wootters Outline Introduction Mathematical Formulation
More informationCompressed Sensing - Near Optimal Recovery of Signals from Highly Incomplete Measurements
Compressed Sensing - Near Optimal Recovery of Signals from Highly Incomplete Measurements Wolfgang Dahmen Institut für Geometrie und Praktische Mathematik RWTH Aachen and IMI, University of Columbia, SC
More informationExplicit Constructions for Compressed Sensing of Sparse Signals
Explicit Constructions for Compressed Sensing of Sparse Signals Piotr Indyk MIT July 12, 2007 1 Introduction Over the recent years, a new approach for obtaining a succinct approximate representation of
More informationMultipath Matching Pursuit
Multipath Matching Pursuit Submitted to IEEE trans. on Information theory Authors: S. Kwon, J. Wang, and B. Shim Presenter: Hwanchol Jang Multipath is investigated rather than a single path for a greedy
More informationCompressed sensing. Or: the equation Ax = b, revisited. Terence Tao. Mahler Lecture Series. University of California, Los Angeles
Or: the equation Ax = b, revisited University of California, Los Angeles Mahler Lecture Series Acquiring signals Many types of real-world signals (e.g. sound, images, video) can be viewed as an n-dimensional
More informationReconstruction from Anisotropic Random Measurements
Reconstruction from Anisotropic Random Measurements Mark Rudelson and Shuheng Zhou The University of Michigan, Ann Arbor Coding, Complexity, and Sparsity Workshop, 013 Ann Arbor, Michigan August 7, 013
More informationCompressed Sensing and Sparse Recovery
ELE 538B: Sparsity, Structure and Inference Compressed Sensing and Sparse Recovery Yuxin Chen Princeton University, Spring 217 Outline Restricted isometry property (RIP) A RIPless theory Compressed sensing
More informationTutorial: Sparse Recovery Using Sparse Matrices. Piotr Indyk MIT
Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup:
More informationNear Optimal Signal Recovery from Random Projections
1 Near Optimal Signal Recovery from Random Projections Emmanuel Candès, California Institute of Technology Multiscale Geometric Analysis in High Dimensions: Workshop # 2 IPAM, UCLA, October 2004 Collaborators:
More informationGeneralized Orthogonal Matching Pursuit- A Review and Some
Generalized Orthogonal Matching Pursuit- A Review and Some New Results Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur, INDIA Table of Contents
More informationThresholds for the Recovery of Sparse Solutions via L1 Minimization
Thresholds for the Recovery of Sparse Solutions via L Minimization David L. Donoho Department of Statistics Stanford University 39 Serra Mall, Sequoia Hall Stanford, CA 9435-465 Email: donoho@stanford.edu
More informationSparse Recovery Using Sparse (Random) Matrices
Sparse Recovery Using Sparse (Random) Matrices Piotr Indyk MIT Joint work with: Radu Berinde, Anna Gilbert, Howard Karloff, Martin Strauss and Milan Ruzic Linear Compression (learning Fourier coeffs, linear
More informationCompressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery
Compressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery Jorge F. Silva and Eduardo Pavez Department of Electrical Engineering Information and Decision Systems Group Universidad
More informationSparse Optimization Lecture: Sparse Recovery Guarantees
Those who complete this lecture will know Sparse Optimization Lecture: Sparse Recovery Guarantees Sparse Optimization Lecture: Sparse Recovery Guarantees Instructor: Wotao Yin Department of Mathematics,
More informationRapidly Computing Sparse Chebyshev and Legendre Coefficient Expansions via SFTs
Rapidly Computing Sparse Chebyshev and Legendre Coefficient Expansions via SFTs Mark Iwen Michigan State University October 18, 2014 Work with Janice (Xianfeng) Hu Graduating in May! M.A. Iwen (MSU) Fast
More informationHeavy Hitters. Piotr Indyk MIT. Lecture 4
Heavy Hitters Piotr Indyk MIT Last Few Lectures Recap (last few lectures) Update a vector x Maintain a linear sketch Can compute L p norm of x (in zillion different ways) Questions: Can we do anything
More informationECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis Lecture 3: Sparse signal recovery: A RIPless analysis of l 1 minimization Yuejie Chi The Ohio State University Page 1 Outline
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationPre-weighted Matching Pursuit Algorithms for Sparse Recovery
Journal of Information & Computational Science 11:9 (214) 2933 2939 June 1, 214 Available at http://www.joics.com Pre-weighted Matching Pursuit Algorithms for Sparse Recovery Jingfei He, Guiling Sun, Jie
More informationSparse Solutions of an Undetermined Linear System
1 Sparse Solutions of an Undetermined Linear System Maddullah Almerdasy New York University Tandon School of Engineering arxiv:1702.07096v1 [math.oc] 23 Feb 2017 Abstract This work proposes a research
More informationTutorial: Sparse Recovery Using Sparse Matrices. Piotr Indyk MIT
Tutorial: Sparse Recovery Using Sparse Matrices Piotr Indyk MIT Problem Formulation (approximation theory, learning Fourier coeffs, linear sketching, finite rate of innovation, compressed sensing...) Setup:
More informationLecture 16: Compressed Sensing
Lecture 16: Compressed Sensing Introduction to Learning and Analysis of Big Data Kontorovich and Sabato (BGU) Lecture 16 1 / 12 Review of Johnson-Lindenstrauss Unsupervised learning technique key insight:
More informationNear-Optimal Sparse Recovery in the L 1 norm
Near-Optimal Sparse Recovery in the L 1 norm Piotr Indyk MIT indyk@mit.edu Milan Ružić ITU Copenhagen milan@itu.dk Abstract We consider the approximate sparse recovery problem, where the goal is to (approximately)
More informationCompressed Sensing: Lecture I. Ronald DeVore
Compressed Sensing: Lecture I Ronald DeVore Motivation Compressed Sensing is a new paradigm for signal/image/function acquisition Motivation Compressed Sensing is a new paradigm for signal/image/function
More informationORTHOGONAL matching pursuit (OMP) is the canonical
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 9, SEPTEMBER 2010 4395 Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property Mark A. Davenport, Member, IEEE, and Michael
More informationAlgorithmic linear dimension reduction in the l 1 norm for sparse vectors
Algorithmic linear dimension reduction in the l norm for sparse vectors A. C. Gilbert, M. J. Strauss, J. A. Tropp, and R. Vershynin Abstract Using a number of different algorithms, we can recover approximately
More informationTowards an Algorithmic Theory of Compressed Sensing
DIMACS Technical Report 2005-25 September 2005 Towards an Algorithmic Theory of Compressed Sensing by Graham Cormode 1 Bell Labs cormode@bell-labs.com S. Muthukrishnan 2,3 Rutgers University muthu@cs.rutgers.edu
More informationIEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER
IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER 2015 1239 Preconditioning for Underdetermined Linear Systems with Sparse Solutions Evaggelia Tsiligianni, StudentMember,IEEE, Lisimachos P. Kondi,
More informationThree Generalizations of Compressed Sensing
Thomas Blumensath School of Mathematics The University of Southampton June, 2010 home prev next page Compressed Sensing and beyond y = Φx + e x R N or x C N x K is K-sparse and x x K 2 is small y R M or
More informationSparse Fourier Transform (lecture 4)
1 / 5 Sparse Fourier Transform (lecture 4) Michael Kapralov 1 1 IBM Watson EPFL St. Petersburg CS Club November 215 2 / 5 Given x C n, compute the Discrete Fourier Transform of x: x i = 1 n x j ω ij, j
More informationCompressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
More informationEE 381V: Large Scale Optimization Fall Lecture 24 April 11
EE 381V: Large Scale Optimization Fall 2012 Lecture 24 April 11 Lecturer: Caramanis & Sanghavi Scribe: Tao Huang 24.1 Review In past classes, we studied the problem of sparsity. Sparsity problem is that
More informationLecture 13 October 6, Covering Numbers and Maurey s Empirical Method
CS 395T: Sublinear Algorithms Fall 2016 Prof. Eric Price Lecture 13 October 6, 2016 Scribe: Kiyeon Jeon and Loc Hoang 1 Overview In the last lecture we covered the lower bound for p th moment (p > 2) and
More informationNear Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing
Near Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas M.Vidyasagar@utdallas.edu www.utdallas.edu/ m.vidyasagar
More informationNoisy Signal Recovery via Iterative Reweighted L1-Minimization
Noisy Signal Recovery via Iterative Reweighted L1-Minimization Deanna Needell UC Davis / Stanford University Asilomar SSC, November 2009 Problem Background Setup 1 Suppose x is an unknown signal in R d.
More informationCompressed Sensing with Very Sparse Gaussian Random Projections
Compressed Sensing with Very Sparse Gaussian Random Projections arxiv:408.504v stat.me] Aug 04 Ping Li Department of Statistics and Biostatistics Department of Computer Science Rutgers University Piscataway,
More informationThe Analysis Cosparse Model for Signals and Images
The Analysis Cosparse Model for Signals and Images Raja Giryes Computer Science Department, Technion. The research leading to these results has received funding from the European Research Council under
More informationInverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France
Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Structure of the tutorial Session 1: Introduction to inverse problems & sparse
More informationCombining geometry and combinatorics: a unified approach to sparse signal recovery
Combining geometry and combinatorics: a unified approach to sparse signal recovery R. Berinde, A. C. Gilbert, P. Indyk, H. Karloff, and M. J. Strauss Abstract There are two main algorithmic approaches
More informationElaine T. Hale, Wotao Yin, Yin Zhang
, Wotao Yin, Yin Zhang Department of Computational and Applied Mathematics Rice University McMaster University, ICCOPT II-MOPTA 2007 August 13, 2007 1 with Noise 2 3 4 1 with Noise 2 3 4 1 with Noise 2
More informationCS 229r: Algorithms for Big Data Fall Lecture 19 Nov 5
CS 229r: Algorithms for Big Data Fall 215 Prof. Jelani Nelson Lecture 19 Nov 5 Scribe: Abdul Wasay 1 Overview In the last lecture, we started discussing the problem of compressed sensing where we are given
More informationCompressed Sensing: Extending CLEAN and NNLS
Compressed Sensing: Extending CLEAN and NNLS Ludwig Schwardt SKA South Africa (KAT Project) Calibration & Imaging Workshop Socorro, NM, USA 31 March 2009 Outline 1 Compressed Sensing (CS) Introduction
More informationA tutorial on sparse modeling. Outline:
A tutorial on sparse modeling. Outline: 1. Why? 2. What? 3. How. 4. no really, why? Sparse modeling is a component in many state of the art signal processing and machine learning tasks. image processing
More informationMATCHING PURSUIT WITH STOCHASTIC SELECTION
2th European Signal Processing Conference (EUSIPCO 22) Bucharest, Romania, August 27-3, 22 MATCHING PURSUIT WITH STOCHASTIC SELECTION Thomas Peel, Valentin Emiya, Liva Ralaivola Aix-Marseille Université
More informationAN INTRODUCTION TO COMPRESSIVE SENSING
AN INTRODUCTION TO COMPRESSIVE SENSING Rodrigo B. Platte School of Mathematical and Statistical Sciences APM/EEE598 Reverse Engineering of Complex Dynamical Networks OUTLINE 1 INTRODUCTION 2 INCOHERENCE
More informationCOMPRESSED SENSING IN PYTHON
COMPRESSED SENSING IN PYTHON Sercan Yıldız syildiz@samsi.info February 27, 2017 OUTLINE A BRIEF INTRODUCTION TO COMPRESSED SENSING A BRIEF INTRODUCTION TO CVXOPT EXAMPLES A Brief Introduction to Compressed
More informationSignal Recovery From Incomplete and Inaccurate Measurements via Regularized Orthogonal Matching Pursuit
Signal Recovery From Incomplete and Inaccurate Measurements via Regularized Orthogonal Matching Pursuit Deanna Needell and Roman Vershynin Abstract We demonstrate a simple greedy algorithm that can reliably
More informationCOMPRESSED SENSING AND BEST k-term APPROXIMATION
JOURNAL OF THE AMERICAN MATHEMATICAL SOCIETY Volume 22, Number 1, January 2009, Pages 211 231 S 0894-0347(08)00610-3 Article electronically published on July 31, 2008 COMPRESSED SENSING AND BEST k-term
More informationOrthogonal Matching Pursuit for Sparse Signal Recovery With Noise
Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published
More informationRecovery Guarantees for Rank Aware Pursuits
BLANCHARD AND DAVIES: RECOVERY GUARANTEES FOR RANK AWARE PURSUITS 1 Recovery Guarantees for Rank Aware Pursuits Jeffrey D. Blanchard and Mike E. Davies Abstract This paper considers sufficient conditions
More informationStrengthened Sobolev inequalities for a random subspace of functions
Strengthened Sobolev inequalities for a random subspace of functions Rachel Ward University of Texas at Austin April 2013 2 Discrete Sobolev inequalities Proposition (Sobolev inequality for discrete images)
More informationSPARSE signal representations have gained popularity in recent
6958 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 10, OCTOBER 2011 Blind Compressed Sensing Sivan Gleichman and Yonina C. Eldar, Senior Member, IEEE Abstract The fundamental principle underlying
More informationOptimization for Compressed Sensing
Optimization for Compressed Sensing Robert J. Vanderbei 2014 March 21 Dept. of Industrial & Systems Engineering University of Florida http://www.princeton.edu/ rvdb Lasso Regression The problem is to solve
More informationCompressive Sensing Theory and L1-Related Optimization Algorithms
Compressive Sensing Theory and L1-Related Optimization Algorithms Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, USA CAAM Colloquium January 26, 2009 Outline:
More informationSparse analysis Lecture II: Hardness results for sparse approximation problems
Sparse analysis Lecture II: Hardness results for sparse approximation problems Anna C. Gilbert Department of Mathematics University of Michigan Sparse Problems Exact. Given a vector x R d and a complete
More informationGradient Descent with Sparsification: An iterative algorithm for sparse recovery with restricted isometry property
: An iterative algorithm for sparse recovery with restricted isometry property Rahul Garg grahul@us.ibm.com Rohit Khandekar rohitk@us.ibm.com IBM T. J. Watson Research Center, 0 Kitchawan Road, Route 34,
More informationLecture 16 Oct. 26, 2017
Sketching Algorithms for Big Data Fall 2017 Prof. Piotr Indyk Lecture 16 Oct. 26, 2017 Scribe: Chi-Ning Chou 1 Overview In the last lecture we constructed sparse RIP 1 matrix via expander and showed that
More informationExact Signal Recovery from Sparsely Corrupted Measurements through the Pursuit of Justice
Exact Signal Recovery from Sparsely Corrupted Measurements through the Pursuit of Justice Jason N. Laska, Mark A. Davenport, Richard G. Baraniuk Department of Electrical and Computer Engineering Rice University
More informationRecovering overcomplete sparse representations from structured sensing
Recovering overcomplete sparse representations from structured sensing Deanna Needell Claremont McKenna College Feb. 2015 Support: Alfred P. Sloan Foundation and NSF CAREER #1348721. Joint work with Felix
More informationCompressive Sensing of Streams of Pulses
Compressive Sensing of Streams of Pulses Chinmay Hegde and Richard G. Baraniuk Department of Electrical and Computer Engineering Rice University Abstract Compressive Sensing (CS) has developed as an enticing
More informationSparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images
Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images Alfredo Nava-Tudela ant@umd.edu John J. Benedetto Department of Mathematics jjb@umd.edu Abstract In this project we are
More informationOptimal Deterministic Compressed Sensing Matrices
Optimal Deterministic Compressed Sensing Matrices Arash Saber Tehrani email: saberteh@usc.edu Alexandros G. Dimakis email: dimakis@usc.edu Giuseppe Caire email: caire@usc.edu Abstract We present the first
More informationEfficient Inverse Cholesky Factorization for Alamouti Matrices in G-STBC and Alamouti-like Matrices in OMP
Efficient Inverse Cholesky Factorization for Alamouti Matrices in G-STBC and Alamouti-like Matrices in OMP Hufei Zhu, Ganghua Yang Communications Technology Laboratory Huawei Technologies Co Ltd, P R China
More informationSparse Legendre expansions via l 1 minimization
Sparse Legendre expansions via l 1 minimization Rachel Ward, Courant Institute, NYU Joint work with Holger Rauhut, Hausdorff Center for Mathematics, Bonn, Germany. June 8, 2010 Outline Sparse recovery
More informationINDUSTRIAL MATHEMATICS INSTITUTE. B.S. Kashin and V.N. Temlyakov. IMI Preprint Series. Department of Mathematics University of South Carolina
INDUSTRIAL MATHEMATICS INSTITUTE 2007:08 A remark on compressed sensing B.S. Kashin and V.N. Temlyakov IMI Preprint Series Department of Mathematics University of South Carolina A remark on compressed
More informationCompressed sensing and best k-term approximation
Compressed sensing and best k-term approximation Albert Cohen, Wolfgang Dahmen, and Ronald DeVore July 19, 2006 Abstract Compressed sensing is a new concept in signal processing where one seeks to minimize
More informationThe Fundamentals of Compressive Sensing
The Fundamentals of Compressive Sensing Mark A. Davenport Georgia Institute of Technology School of Electrical and Computer Engineering Sensor Explosion Data Deluge Digital Revolution If we sample a signal
More informationof Orthogonal Matching Pursuit
A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit Qun Mo arxiv:50.0708v [cs.it] 8 Jan 205 Abstract We shall show that if the restricted isometry constant (RIC) δ s+ (A) of the measurement
More informationDetecting Sparse Structures in Data in Sub-Linear Time: A group testing approach
Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach Boaz Nadler The Weizmann Institute of Science Israel Joint works with Inbal Horev, Ronen Basri, Meirav Galun and Ery Arias-Castro
More informationSparse Expander-like Real-valued Projection (SERP) matrices for compressed sensing
Sparse Expander-like Real-valued Projection (SERP) matrices for compressed sensing Abdolreza Abdolhosseini Moghadam and Hayder Radha Department of Electrical and Computer Engineering, Michigan State University,
More informationInstance Optimal Decoding by Thresholding in Compressed Sensing
Instance Optimal Decoding by Thresholding in Compressed Sensing Albert Cohen, Wolfgang Dahmen, and Ronald DeVore November 1, 2008 Abstract Compressed Sensing sees to capture a discrete signal x IR N with
More informationIntroduction How it works Theory behind Compressed Sensing. Compressed Sensing. Huichao Xue. CS3750 Fall 2011
Compressed Sensing Huichao Xue CS3750 Fall 2011 Table of Contents Introduction From News Reports Abstract Definition How it works A review of L 1 norm The Algorithm Backgrounds for underdetermined linear
More informationWavelet decomposition of data streams. by Dragana Veljkovic
Wavelet decomposition of data streams by Dragana Veljkovic Motivation Continuous data streams arise naturally in: telecommunication and internet traffic retail and banking transactions web server log records
More informationLecture 22: More On Compressed Sensing
Lecture 22: More On Compressed Sensing Scribed by Eric Lee, Chengrun Yang, and Sebastian Ament Nov. 2, 207 Recap and Introduction Basis pursuit was the method of recovering the sparsest solution to an
More information