A Tutorial on Compressive Sensing. Simon Foucart Drexel University / University of Georgia
|
|
- Delphia Fletcher
- 5 years ago
- Views:
Transcription
1 A Tutorial on Compressive Sensing Simon Foucart Drexel University / University of Georgia CIMPA13 New Trends in Applied Harmonic Analysis Mar del Plata, Argentina, 5-16 August 2013
2 This minicourse acts as an invitation to the elegant theory of Compressive Sensing. It aims at giving a solid overview of the fundamental mathematical aspects. Its content is partly based on a recent textbook coauthored with Holger Rauhut:
3 Part 1: The Standard Problem and First Algorithms The lecture introduces the question of sparse recovery, establishes its theoretical limits, and presents an algorithm achieving these limits in an idealized situation. In order to treat more realistic situations, other algorithms are necessary. Basis Pursuit (that is, l 1 -minimization) and Orthogonal Matching Pursuit make their first appearance. Their success is proved using the concept of coherence of a matrix.
4 Keywords
5 Keywords Sparsity Essential
6 Keywords Sparsity Essential Randomness Nothing better so far (measurement process)
7 Keywords Sparsity Essential Randomness Nothing better so far (measurement process) Optimization Preferred, but competitive alternatives are available (reconstruction process)
8 The Standard Compressive Sensing Problem
9 The Standard Compressive Sensing Problem x : unknown signal of interest in K N
10 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m
11 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N,
12 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x
13 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x = card { j {1,..., N} : x j 0 }.
14 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x = card { j {1,..., N} : x j 0 }. Find concrete sensing/recovery protocols,
15 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x = card { j {1,..., N} : x j 0 }. Find concrete sensing/recovery protocols, i.e., find measurement matrices A : x K N y K m
16 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x = card { j {1,..., N} : x j 0 }. Find concrete sensing/recovery protocols, i.e., find measurement matrices A : x K N y K m reconstruction maps : y K m x K N
17 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x = card { j {1,..., N} : x j 0 }. Find concrete sensing/recovery protocols, i.e., find measurement matrices A : x K N y K m reconstruction maps : y K m x K N such that (Ax) = x for any s-sparse vector x K N.
18 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x = card { j {1,..., N} : x j 0 }. Find concrete sensing/recovery protocols, i.e., find measurement matrices A : x K N y K m reconstruction maps : y K m x K N such that (Ax) = x for any s-sparse vector x K N. In realistic situations, two issues to consider:
19 The Standard Compressive Sensing Problem x : unknown signal of interest in K N y : measurement vector in K m with m N, s : sparsity of x = card { j {1,..., N} : x j 0 }. Find concrete sensing/recovery protocols, i.e., find measurement matrices A : x K N y K m reconstruction maps : y K m x K N such that (Ax) = x for any s-sparse vector x K N. In realistic situations, two issues to consider: Stability: x not sparse but compressible, Robustness: measurement error in y = Ax + e.
20 A Selection of Applications
21 A Selection of Applications Magnetic resonance imaging Figure: Left: traditional MRI reconstruction; Right: compressive sensing reconstruction (courtesy of M. Lustig and S. Vasanawala)
22 A Selection of Applications Magnetic resonance imaging Sampling theory Figure: Time-domain signal with 16 samples.
23 A Selection of Applications Magnetic resonance imaging Sampling theory Error correction
24 A Selection of Applications Magnetic resonance imaging Sampling theory Error correction and many more...
25 l 0 -Minimization Since x p p := N j=1 x j p p 0 N j=1 1 {xj 0},
26 l 0 -Minimization Since x p p := N j=1 x j p p 0 N j=1 the notation x 0 [sic] has become usual for 1 {xj 0}, x 0 := card(supp(x)), where supp(x) := {j [N] : x j 0}.
27 l 0 -Minimization Since x p p := N j=1 x j p p 0 N j=1 the notation x 0 [sic] has become usual for 1 {xj 0}, x 0 := card(supp(x)), where supp(x) := {j [N] : x j 0}. For an s-sparse x K N, observe the equivalence of
28 l 0 -Minimization Since x p p := N j=1 x j p p 0 N j=1 the notation x 0 [sic] has become usual for 1 {xj 0}, x 0 := card(supp(x)), where supp(x) := {j [N] : x j 0}. For an s-sparse x K N, observe the equivalence of x is the unique s-sparse solution of Az = y with y = Ax,
29 l 0 -Minimization Since x p p := N j=1 x j p p 0 N j=1 the notation x 0 [sic] has become usual for 1 {xj 0}, x 0 := card(supp(x)), where supp(x) := {j [N] : x j 0}. For an s-sparse x K N, observe the equivalence of x is the unique s-sparse solution of Az = y with y = Ax, x can be reconstructed as the unique solution of (P 0 ) minimize z K N z 0 subject to Az = y.
30 l 0 -Minimization Since x p p := N j=1 x j p p 0 N j=1 the notation x 0 [sic] has become usual for 1 {xj 0}, x 0 := card(supp(x)), where supp(x) := {j [N] : x j 0}. For an s-sparse x K N, observe the equivalence of x is the unique s-sparse solution of Az = y with y = Ax, x can be reconstructed as the unique solution of (P 0 ) minimize z K N z 0 subject to Az = y. This is a combinatorial problem, NP-hard in general.
31 Minimal Number of Measurements
32 Minimal Number of Measurements Given A K m N, the following are equivalent:
33 Minimal Number of Measurements Given A K m N, the following are equivalent: 1. Every s-sparse x is the unique s-sparse solution of Az = Ax,
34 Minimal Number of Measurements Given A K m N, the following are equivalent: 1. Every s-sparse x is the unique s-sparse solution of Az = Ax, 2. ker A {z K N : z 0 2s} = {0},
35 Minimal Number of Measurements Given A K m N, the following are equivalent: 1. Every s-sparse x is the unique s-sparse solution of Az = Ax, 2. ker A {z K N : z 0 2s} = {0}, 3. For any S [N] with card(s) 2s, the matrix A S is injective,
36 Minimal Number of Measurements Given A K m N, the following are equivalent: 1. Every s-sparse x is the unique s-sparse solution of Az = Ax, 2. ker A {z K N : z 0 2s} = {0}, 3. For any S [N] with card(s) 2s, the matrix A S is injective, 4. Every set of 2s columns of A is linearly independent.
37 Minimal Number of Measurements Given A K m N, the following are equivalent: 1. Every s-sparse x is the unique s-sparse solution of Az = Ax, 2. ker A {z K N : z 0 2s} = {0}, 3. For any S [N] with card(s) 2s, the matrix A S is injective, 4. Every set of 2s columns of A is linearly independent. As a consequence, exact recovery of every s-sparse vector forces m 2s.
38 Minimal Number of Measurements Given A K m N, the following are equivalent: 1. Every s-sparse x is the unique s-sparse solution of Az = Ax, 2. ker A {z K N : z 0 2s} = {0}, 3. For any S [N] with card(s) 2s, the matrix A S is injective, 4. Every set of 2s columns of A is linearly independent. As a consequence, exact recovery of every s-sparse vector forces m 2s. This can be achieved using partial Vandermonde matrices.
39 Exact s-sparse Recovery from 2s Fourier Measurements
40 Exact s-sparse Recovery from 2s Fourier Measurements Identify an s-sparse x C N with a function x on {0, 1,..., N 1} with support S, card(s) = s.
41 Exact s-sparse Recovery from 2s Fourier Measurements Identify an s-sparse x C N with a function x on {0, 1,..., N 1} with support S, card(s) = s. Consider the 2s Fourier coefficients ˆx(j) = N 1 k=0 x(k)e i2πjk/n, 0 j 2s 1.
42 Exact s-sparse Recovery from 2s Fourier Measurements Identify an s-sparse x C N with a function x on {0, 1,..., N 1} with support S, card(s) = s. Consider the 2s Fourier coefficients ˆx(j) = N 1 k=0 x(k)e i2πjk/n, 0 j 2s 1. Consider a trigonometric polynomial vanishing exactly on S, i.e., p(t) := k S ( 1 e i2πk/n e i2πt/n).
43 Exact s-sparse Recovery from 2s Fourier Measurements Identify an s-sparse x C N with a function x on {0, 1,..., N 1} with support S, card(s) = s. Consider the 2s Fourier coefficients ˆx(j) = N 1 k=0 x(k)e i2πjk/n, 0 j 2s 1. Consider a trigonometric polynomial vanishing exactly on S, i.e., p(t) := k S ( 1 e i2πk/n e i2πt/n). Since p x 0, discrete convolution gives 0 = (ˆp ˆx)(j) = N 1 k=0 ˆp(k)ˆx(j k), 0 j N 1.
44 Exact s-sparse Recovery from 2s Fourier Measurements Identify an s-sparse x C N with a function x on {0, 1,..., N 1} with support S, card(s) = s. Consider the 2s Fourier coefficients ˆx(j) = N 1 k=0 x(k)e i2πjk/n, 0 j 2s 1. Consider a trigonometric polynomial vanishing exactly on S, i.e., p(t) := k S ( 1 e i2πk/n e i2πt/n). Since p x 0, discrete convolution gives 0 = (ˆp ˆx)(j) = N 1 k=0 ˆp(k)ˆx(j k), 0 j N 1. Note that ˆp(0) = 1 and that ˆp(k) = 0 for k > s.
45 Exact s-sparse Recovery from 2s Fourier Measurements Identify an s-sparse x C N with a function x on {0, 1,..., N 1} with support S, card(s) = s. Consider the 2s Fourier coefficients ˆx(j) = N 1 k=0 x(k)e i2πjk/n, 0 j 2s 1. Consider a trigonometric polynomial vanishing exactly on S, i.e., p(t) := k S ( 1 e i2πk/n e i2πt/n). Since p x 0, discrete convolution gives 0 = (ˆp ˆx)(j) = N 1 k=0 ˆp(k)ˆx(j k), 0 j N 1. Note that ˆp(0) = 1 and that ˆp(k) = 0 for k > s. The equations s,..., 2s 1 translate into a Toeplitz system with unknowns ˆp(1),..., ˆp(s).
46 Exact s-sparse Recovery from 2s Fourier Measurements Identify an s-sparse x C N with a function x on {0, 1,..., N 1} with support S, card(s) = s. Consider the 2s Fourier coefficients ˆx(j) = N 1 k=0 x(k)e i2πjk/n, 0 j 2s 1. Consider a trigonometric polynomial vanishing exactly on S, i.e., p(t) := k S ( 1 e i2πk/n e i2πt/n). Since p x 0, discrete convolution gives 0 = (ˆp ˆx)(j) = N 1 k=0 ˆp(k)ˆx(j k), 0 j N 1. Note that ˆp(0) = 1 and that ˆp(k) = 0 for k > s. The equations s,..., 2s 1 translate into a Toeplitz system with unknowns ˆp(1),..., ˆp(s). This determines ˆp, hence p, then S, and finally x.
47 Optimization and Greedy Strategies
48 l 1 -Minimization (Basis Pursuit)
49 l 1 -Minimization (Basis Pursuit) Replace (P 0 ) by (P 1 ) minimize z K N z 1 subject to Az = y.
50 l 1 -Minimization (Basis Pursuit) Replace (P 0 ) by (P 1 ) minimize z K N z 1 subject to Az = y. Geometric intuition
51 l 1 -Minimization (Basis Pursuit) Replace (P 0 ) by (P 1 ) minimize z K N z 1 subject to Az = y. Geometric intuition Unique l 1 -minimizers are at most m-sparse (when K = R)
52 l 1 -Minimization (Basis Pursuit) Replace (P 0 ) by (P 1 ) minimize z K N z 1 subject to Az = y. Geometric intuition Unique l 1 -minimizers are at most m-sparse (when K = R) Convex optimization program, hence solvable in practice
53 l 1 -Minimization (Basis Pursuit) Replace (P 0 ) by (P 1 ) minimize z K N z 1 subject to Az = y. Geometric intuition Unique l 1 -minimizers are at most m-sparse (when K = R) Convex optimization program, hence solvable in practice In the real setting, recast as the linear optimization program minimize c,z R N N c j subject to Az = y and c j z j c j. j=1
54 l 1 -Minimization (Basis Pursuit) Replace (P 0 ) by (P 1 ) minimize z K N z 1 subject to Az = y. Geometric intuition Unique l 1 -minimizers are at most m-sparse (when K = R) Convex optimization program, hence solvable in practice In the real setting, recast as the linear optimization program minimize c,z R N N c j subject to Az = y and c j z j c j. j=1 In the complex setting, recast as a second order cone program
55 Basis Pursuit Null Space Property
56 Basis Pursuit Null Space Property 1 (Ax) = x for every vector x supported on S if and only if
57 Basis Pursuit Null Space Property 1 (Ax) = x for every vector x supported on S if and only if (NSP) u S 1 < u S 1, all u ker A \ {0}.
58 Basis Pursuit Null Space Property 1 (Ax) = x for every vector x supported on S if and only if (NSP) u S 1 < u S 1, all u ker A \ {0}. For real measurement matrices,
59 Basis Pursuit Null Space Property 1 (Ax) = x for every vector x supported on S if and only if (NSP) u S 1 < u S 1, all u ker A \ {0}. For real measurement matrices, real and complex NSPs read
60 Basis Pursuit Null Space Property 1 (Ax) = x for every vector x supported on S if and only if (NSP) u S 1 < u S 1, all u ker A \ {0}. For real measurement matrices, real and complex NSPs read u j < u l, all u ker R A \ {0}, j S l S
61 Basis Pursuit Null Space Property 1 (Ax) = x for every vector x supported on S if and only if (NSP) u S 1 < u S 1, all u ker A \ {0}. For real measurement matrices, real and complex NSPs read u j < u l, all u ker R A \ {0}, j S l S vj 2 + wj 2 < vl 2 + w l 2, all (v, w) (ker R A) 2 \ {0}. j S l S
62 Basis Pursuit Null Space Property 1 (Ax) = x for every vector x supported on S if and only if (NSP) u S 1 < u S 1, all u ker A \ {0}. For real measurement matrices, real and complex NSPs read u j < u l, all u ker R A \ {0}, j S l S vj 2 + wj 2 < vl 2 + w l 2, all (v, w) (ker R A) 2 \ {0}. j S l S Real and complex NSPs are in fact equivalent.
63 Orthogonal Matching Pursuit
64 Orthogonal Matching Pursuit Starting with S 0 = and x 0 = 0, iterate (OMP 1 ) (OMP 2 ) S n+1 = S n { j n+1 { := argmax (A (y Ax n )) j }}, j [N] x n+1 = argmin z C N { y Az 2, supp(z) S n+1}.
65 Orthogonal Matching Pursuit Starting with S 0 = and x 0 = 0, iterate S n+1 = S n { j n+1 { (OMP := argmax (A (y Ax n )) j }} 1 ), j [N] (OMP 2 ) x n+1 = argmin z C N { y Az 2, supp(z) S n+1}. The norm of the residual decreases according to y Ax n y Ax n 2 2 (A (y Ax n )) j n+1 2.
66 Orthogonal Matching Pursuit Starting with S 0 = and x 0 = 0, iterate S n+1 = S n { j n+1 { (OMP := argmax (A (y Ax n )) j }} 1 ), j [N] (OMP 2 ) x n+1 = argmin z C N { y Az 2, supp(z) S n+1}. The norm of the residual decreases according to y Ax n y Ax n 2 2 (A (y Ax n )) j n+1 2. Every vector x 0 supported on S, card(s) = s, is recovered from y = Ax after at most s iterations of OMP if and only if A S is injective and (ERC) max j S (A r) j > max (A r) l l S for all r 0 { Az, supp(z) S }.
67 First Recovery Guarantees
68 Coherence
69 Coherence For a matrix with l 2 -normalized columns a 1,..., a N, define µ := max a i, a j. i j
70 Coherence For a matrix with l 2 -normalized columns a 1,..., a N, define µ := max a i, a j. i j As a rule, the smaller the coherence, the better.
71 Coherence For a matrix with l 2 -normalized columns a 1,..., a N, define µ := max a i, a j. i j As a rule, the smaller the coherence, the better. However, the Welch bound reads N m µ m(n 1).
72 Coherence For a matrix with l 2 -normalized columns a 1,..., a N, define µ := max a i, a j. i j As a rule, the smaller the coherence, the better. However, the Welch bound reads N m µ m(n 1). Welch bound achieved at and only at equiangular tight frames
73 Coherence For a matrix with l 2 -normalized columns a 1,..., a N, define µ := max a i, a j. i j As a rule, the smaller the coherence, the better. However, the Welch bound reads N m µ m(n 1). Welch bound achieved at and only at equiangular tight frames Deterministic matrices with coherence µ c/ m exist
74 Recovery Conditions using Coherence
75 Recovery Conditions using Coherence Every s-sparse x C N is recovered from y = Ax C m via at most s iterations of OMP provided µ < 1 2s 1.
76 Recovery Conditions using Coherence Every s-sparse x C N is recovered from y = Ax C m via at most s iterations of OMP provided µ < 1 2s 1. Every s-sparse x C N is recovered from y = Ax C m via Basis Pursuit provided µ < 1 2s 1.
77 Recovery Conditions using Coherence Every s-sparse x C N is recovered from y = Ax C m via at most s iterations of OMP provided µ < 1 2s 1. Every s-sparse x C N is recovered from y = Ax C m via Basis Pursuit provided µ < 1 2s 1. In fact, the Exact Recovery Condition can be rephrased as A S A S 1 1 < 1, and this implies the Null Space Property.
78 Summary
79 Summary The coherence conditions for s-sparse recovery are of the type µ c s,
80 Summary The coherence conditions for s-sparse recovery are of the type µ c s, while the Welch bound (for N 2m, say) reads µ c m.
81 Summary The coherence conditions for s-sparse recovery are of the type µ c s, while the Welch bound (for N 2m, say) reads µ c m. Therefore, arguments based on coherence necessitate m c s 2.
82 Summary The coherence conditions for s-sparse recovery are of the type µ c s, while the Welch bound (for N 2m, say) reads µ c m. Therefore, arguments based on coherence necessitate m c s 2. This is far from the ideal linear scaling of m in s...
83 Summary The coherence conditions for s-sparse recovery are of the type µ c s, while the Welch bound (for N 2m, say) reads µ c m. Therefore, arguments based on coherence necessitate m c s 2. This is far from the ideal linear scaling of m in s... Next, we will introduce new tools to break this quadratic barrier (with random matrices).
List of Errata for the Book A Mathematical Introduction to Compressive Sensing
List of Errata for the Book A Mathematical Introduction to Compressive Sensing Simon Foucart and Holger Rauhut This list was last updated on September 22, 2018. If you see further errors, please send us
More informationSparse solutions of underdetermined systems
Sparse solutions of underdetermined systems I-Liang Chern September 22, 2016 1 / 16 Outline Sparsity and Compressibility: the concept for measuring sparsity and compressibility of data Minimum measurements
More informationIntroduction to Compressed Sensing
Introduction to Compressed Sensing Alejandro Parada, Gonzalo Arce University of Delaware August 25, 2016 Motivation: Classical Sampling 1 Motivation: Classical Sampling Issues Some applications Radar Spectral
More informationStability and Robustness of Weak Orthogonal Matching Pursuits
Stability and Robustness of Weak Orthogonal Matching Pursuits Simon Foucart, Drexel University Abstract A recent result establishing, under restricted isometry conditions, the success of sparse recovery
More informationAn Introduction to Sparse Approximation
An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,
More informationTwo added structures in sparse recovery: nonnegativity and disjointedness. Simon Foucart University of Georgia
Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart University of Georgia Semester Program on High-Dimensional Approximation ICERM 7 October 2014 Part I: Nonnegative
More informationChapter 2 Sparse Solutions of Underdetermined Systems
Chapter 2 Sparse Solutions of Underdetermined Systems In this chapter, we define the notions of vector sparsity and compressibility, and we establish some related inequalities used throughout the book
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationConstructing Explicit RIP Matrices and the Square-Root Bottleneck
Constructing Explicit RIP Matrices and the Square-Root Bottleneck Ryan Cinoman July 18, 2018 Ryan Cinoman Constructing Explicit RIP Matrices July 18, 2018 1 / 36 Outline 1 Introduction 2 Restricted Isometry
More informationNotes. Simon Foucart
Notes on Compressed Sensing for Math 394 Simon Foucart Spring 2009 Foreword I have started to put these notes together in December 2008. They are intended for a graduate course on Compressed Sensing in
More informationGeneralized Orthogonal Matching Pursuit- A Review and Some
Generalized Orthogonal Matching Pursuit- A Review and Some New Results Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur, INDIA Table of Contents
More information6 Compressed Sensing and Sparse Recovery
6 Compressed Sensing and Sparse Recovery Most of us have noticed how saving an image in JPEG dramatically reduces the space it occupies in our hard drives as oppose to file types that save the pixel value
More informationLecture 22: More On Compressed Sensing
Lecture 22: More On Compressed Sensing Scribed by Eric Lee, Chengrun Yang, and Sebastian Ament Nov. 2, 207 Recap and Introduction Basis pursuit was the method of recovering the sparsest solution to an
More informationInverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France
Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Structure of the tutorial Session 1: Introduction to inverse problems & sparse
More informationSparse analysis Lecture III: Dictionary geometry and greedy algorithms
Sparse analysis Lecture III: Dictionary geometry and greedy algorithms Anna C. Gilbert Department of Mathematics University of Michigan Intuition from ONB Key step in algorithm: r, ϕ j = x c i ϕ i, ϕ j
More informationGreedy Signal Recovery and Uniform Uncertainty Principles
Greedy Signal Recovery and Uniform Uncertainty Principles SPIE - IE 2008 Deanna Needell Joint work with Roman Vershynin UC Davis, January 2008 Greedy Signal Recovery and Uniform Uncertainty Principles
More informationCompressive Sensing and Beyond
Compressive Sensing and Beyond Sohail Bahmani Gerorgia Tech. Signal Processing Compressed Sensing Signal Models Classics: bandlimited The Sampling Theorem Any signal with bandwidth B can be recovered
More informationStability and robustness of l 1 -minimizations with Weibull matrices and redundant dictionaries
Stability and robustness of l 1 -minimizations with Weibull matrices and redundant dictionaries Simon Foucart, Drexel University Abstract We investigate the recovery of almost s-sparse vectors x C N from
More informationNew Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit
New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence
More informationCompressed Sensing. 1 Introduction. 2 Design of Measurement Matrices
Compressed Sensing Yonina C. Eldar Electrical Engineering Department, Technion-Israel Institute of Technology, Haifa, Israel, 32000 1 Introduction Compressed sensing (CS) is an exciting, rapidly growing
More informationStrengthened Sobolev inequalities for a random subspace of functions
Strengthened Sobolev inequalities for a random subspace of functions Rachel Ward University of Texas at Austin April 2013 2 Discrete Sobolev inequalities Proposition (Sobolev inequality for discrete images)
More informationRecovering overcomplete sparse representations from structured sensing
Recovering overcomplete sparse representations from structured sensing Deanna Needell Claremont McKenna College Feb. 2015 Support: Alfred P. Sloan Foundation and NSF CAREER #1348721. Joint work with Felix
More informationMLCC 2018 Variable Selection and Sparsity. Lorenzo Rosasco UNIGE-MIT-IIT
MLCC 2018 Variable Selection and Sparsity Lorenzo Rosasco UNIGE-MIT-IIT Outline Variable Selection Subset Selection Greedy Methods: (Orthogonal) Matching Pursuit Convex Relaxation: LASSO & Elastic Net
More informationLecture: Introduction to Compressed Sensing Sparse Recovery Guarantees
Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html Acknowledgement: this slides is based on Prof. Emmanuel Candes and Prof. Wotao Yin
More informationSensing systems limited by constraints: physical size, time, cost, energy
Rebecca Willett Sensing systems limited by constraints: physical size, time, cost, energy Reduce the number of measurements needed for reconstruction Higher accuracy data subject to constraints Original
More informationInverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France.
Inverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Overview of the course Introduction sparsity & data compression inverse problems
More informationSparse Legendre expansions via l 1 minimization
Sparse Legendre expansions via l 1 minimization Rachel Ward, Courant Institute, NYU Joint work with Holger Rauhut, Hausdorff Center for Mathematics, Bonn, Germany. June 8, 2010 Outline Sparse recovery
More informationUniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit
Uniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit arxiv:0707.4203v2 [math.na] 14 Aug 2007 Deanna Needell Department of Mathematics University of California,
More informationSparsity and Compressed Sensing
Sparsity and Compressed Sensing Jalal Fadili Normandie Université-ENSICAEN, GREYC Mathematical coffees 2017 Recap: linear inverse problems Dictionary Sensing Sensing Sensing = y m 1 H m n y y 2 R m H A
More informationExponential decay of reconstruction error from binary measurements of sparse signals
Exponential decay of reconstruction error from binary measurements of sparse signals Deanna Needell Joint work with R. Baraniuk, S. Foucart, Y. Plan, and M. Wootters Outline Introduction Mathematical Formulation
More informationCompressed sensing. Or: the equation Ax = b, revisited. Terence Tao. Mahler Lecture Series. University of California, Los Angeles
Or: the equation Ax = b, revisited University of California, Los Angeles Mahler Lecture Series Acquiring signals Many types of real-world signals (e.g. sound, images, video) can be viewed as an n-dimensional
More informationAlgorithms for sparse analysis Lecture I: Background on sparse approximation
Algorithms for sparse analysis Lecture I: Background on sparse approximation Anna C. Gilbert Department of Mathematics University of Michigan Tutorial on sparse approximations and algorithms Compress data
More informationGREEDY SIGNAL RECOVERY REVIEW
GREEDY SIGNAL RECOVERY REVIEW DEANNA NEEDELL, JOEL A. TROPP, ROMAN VERSHYNIN Abstract. The two major approaches to sparse recovery are L 1-minimization and greedy methods. Recently, Needell and Vershynin
More informationRecent Developments in Compressed Sensing
Recent Developments in Compressed Sensing M. Vidyasagar Distinguished Professor, IIT Hyderabad m.vidyasagar@iith.ac.in, www.iith.ac.in/ m vidyasagar/ ISL Seminar, Stanford University, 19 April 2018 Outline
More informationElaine T. Hale, Wotao Yin, Yin Zhang
, Wotao Yin, Yin Zhang Department of Computational and Applied Mathematics Rice University McMaster University, ICCOPT II-MOPTA 2007 August 13, 2007 1 with Noise 2 3 4 1 with Noise 2 3 4 1 with Noise 2
More informationCoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles
CoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles SIAM Student Research Conference Deanna Needell Joint work with Roman Vershynin and Joel Tropp UC Davis, May 2008 CoSaMP: Greedy Signal
More informationBhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego
Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego 1 Outline Course Outline Motivation for Course Sparse Signal Recovery Problem Applications Computational
More informationConstrained optimization
Constrained optimization DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Compressed sensing Convex constrained
More informationThe Analysis Cosparse Model for Signals and Images
The Analysis Cosparse Model for Signals and Images Raja Giryes Computer Science Department, Technion. The research leading to these results has received funding from the European Research Council under
More informationSparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery
Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan Connection between... Sparse Approximation and Compressed
More informationSPARSE signal representations have gained popularity in recent
6958 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 10, OCTOBER 2011 Blind Compressed Sensing Sivan Gleichman and Yonina C. Eldar, Senior Member, IEEE Abstract The fundamental principle underlying
More informationRecovery of Sparse Signals from Noisy Measurements Using an l p -Regularized Least-Squares Algorithm
Recovery of Sparse Signals from Noisy Measurements Using an l p -Regularized Least-Squares Algorithm J. K. Pant, W.-S. Lu, and A. Antoniou University of Victoria August 25, 2011 Compressive Sensing 1 University
More informationCOMPRESSED SENSING IN PYTHON
COMPRESSED SENSING IN PYTHON Sercan Yıldız syildiz@samsi.info February 27, 2017 OUTLINE A BRIEF INTRODUCTION TO COMPRESSED SENSING A BRIEF INTRODUCTION TO CVXOPT EXAMPLES A Brief Introduction to Compressed
More informationHard Thresholding Pursuit Algorithms: Number of Iterations
Hard Thresholding Pursuit Algorithms: Number of Iterations Jean-Luc Bouchot, Simon Foucart, Pawel Hitczenko Abstract The Hard Thresholding Pursuit algorithm for sparse recovery is revisited using a new
More informationSparse linear models
Sparse linear models Optimization-Based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_spring16 Carlos Fernandez-Granda 2/22/2016 Introduction Linear transforms Frequency representation Short-time
More informationDesigning Information Devices and Systems I Discussion 13B
EECS 6A Fall 7 Designing Information Devices and Systems I Discussion 3B. Orthogonal Matching Pursuit Lecture Orthogonal Matching Pursuit (OMP) algorithm: Inputs: A set of m songs, each of length n: S
More informationSparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery
Sparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery Anna C. Gilbert Department of Mathematics University of Michigan Sparse signal recovery measurements:
More informationRobust Sparse Recovery via Non-Convex Optimization
Robust Sparse Recovery via Non-Convex Optimization Laming Chen and Yuantao Gu Department of Electronic Engineering, Tsinghua University Homepage: http://gu.ee.tsinghua.edu.cn/ Email: gyt@tsinghua.edu.cn
More informationInverse problems, Dictionary based Signal Models and Compressed Sensing
Inverse problems, Dictionary based Signal Models and Compressed Sensing Rémi Gribonval METISS project-team (audio signal processing, speech recognition, source separation) INRIA, Rennes, France Ecole d
More informationInterpolation via weighted l 1 -minimization
Interpolation via weighted l 1 -minimization Holger Rauhut RWTH Aachen University Lehrstuhl C für Mathematik (Analysis) Mathematical Analysis and Applications Workshop in honor of Rupert Lasser Helmholtz
More informationSparse Recovery with Pre-Gaussian Random Matrices
Sparse Recovery with Pre-Gaussian Random Matrices Simon Foucart Laboratoire Jacques-Louis Lions Université Pierre et Marie Curie Paris, 75013, France Ming-Jun Lai Department of Mathematics University of
More informationMultipath Matching Pursuit
Multipath Matching Pursuit Submitted to IEEE trans. on Information theory Authors: S. Kwon, J. Wang, and B. Shim Presenter: Hwanchol Jang Multipath is investigated rather than a single path for a greedy
More informationOptimization for Compressed Sensing
Optimization for Compressed Sensing Robert J. Vanderbei 2014 March 21 Dept. of Industrial & Systems Engineering University of Florida http://www.princeton.edu/ rvdb Lasso Regression The problem is to solve
More information5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER /$ IEEE
5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER 2009 Uncertainty Relations for Shift-Invariant Analog Signals Yonina C. Eldar, Senior Member, IEEE Abstract The past several years
More information18.S096: Compressed Sensing and Sparse Recovery
18.S096: Compressed Sensing and Sparse Recovery Topics in Mathematics of Data Science Fall 2015) Afonso S. Bandeira bandeira@mit.edu http://math.mit.edu/~bandeira November 10, 2015 These are lecture notes
More informationBeyond incoherence and beyond sparsity: compressed sensing in the real world
Beyond incoherence and beyond sparsity: compressed sensing in the real world Clarice Poon 1st November 2013 University of Cambridge, UK Applied Functional and Harmonic Analysis Group Head of Group Anders
More informationRapidly Computing Sparse Chebyshev and Legendre Coefficient Expansions via SFTs
Rapidly Computing Sparse Chebyshev and Legendre Coefficient Expansions via SFTs Mark Iwen Michigan State University October 18, 2014 Work with Janice (Xianfeng) Hu Graduating in May! M.A. Iwen (MSU) Fast
More informationKomprimované snímání a LASSO jako metody zpracování vysocedimenzionálních dat
Komprimované snímání a jako metody zpracování vysocedimenzionálních dat Jan Vybíral (Charles University Prague, Czech Republic) November 2014 VUT Brno 1 / 49 Definition and motivation Its use in bioinformatics
More informationOn the coherence barrier and analogue problems in compressed sensing
On the coherence barrier and analogue problems in compressed sensing Clarice Poon University of Cambridge June 1, 2017 Joint work with: Ben Adcock Anders Hansen Bogdan Roman (Simon Fraser) (Cambridge)
More informationBlind Compressed Sensing
1 Blind Compressed Sensing Sivan Gleichman and Yonina C. Eldar, Senior Member, IEEE arxiv:1002.2586v2 [cs.it] 28 Apr 2010 Abstract The fundamental principle underlying compressed sensing is that a signal,
More informationLarge-Scale L1-Related Minimization in Compressive Sensing and Beyond
Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Arizona State University March
More informationStochastic geometry and random matrix theory in CS
Stochastic geometry and random matrix theory in CS IPAM: numerical methods for continuous optimization University of Edinburgh Joint with Bah, Blanchard, Cartis, and Donoho Encoder Decoder pair - Encoder/Decoder
More informationIntroduction to Sparsity. Xudong Cao, Jake Dreamtree & Jerry 04/05/2012
Introduction to Sparsity Xudong Cao, Jake Dreamtree & Jerry 04/05/2012 Outline Understanding Sparsity Total variation Compressed sensing(definition) Exact recovery with sparse prior(l 0 ) l 1 relaxation
More informationSignal Recovery from Permuted Observations
EE381V Course Project Signal Recovery from Permuted Observations 1 Problem Shanshan Wu (sw33323) May 8th, 2015 We start with the following problem: let s R n be an unknown n-dimensional real-valued signal,
More informationZ Algorithmic Superpower Randomization October 15th, Lecture 12
15.859-Z Algorithmic Superpower Randomization October 15th, 014 Lecture 1 Lecturer: Bernhard Haeupler Scribe: Goran Žužić Today s lecture is about finding sparse solutions to linear systems. The problem
More informationUniqueness Conditions for A Class of l 0 -Minimization Problems
Uniqueness Conditions for A Class of l 0 -Minimization Problems Chunlei Xu and Yun-Bin Zhao October, 03, Revised January 04 Abstract. We consider a class of l 0 -minimization problems, which is to search
More informationLecture Notes 9: Constrained Optimization
Optimization-based data analysis Fall 017 Lecture Notes 9: Constrained Optimization 1 Compressed sensing 1.1 Underdetermined linear inverse problems Linear inverse problems model measurements of the form
More informationAN INTRODUCTION TO COMPRESSIVE SENSING
AN INTRODUCTION TO COMPRESSIVE SENSING Rodrigo B. Platte School of Mathematical and Statistical Sciences APM/EEE598 Reverse Engineering of Complex Dynamical Networks OUTLINE 1 INTRODUCTION 2 INCOHERENCE
More informationUncertainty principles and sparse approximation
Uncertainty principles and sparse approximation In this lecture, we will consider the special case where the dictionary Ψ is composed of a pair of orthobases. We will see that our ability to find a sparse
More informationIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 11, NOVEMBER On the Performance of Sparse Recovery
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 11, NOVEMBER 2011 7255 On the Performance of Sparse Recovery Via `p-minimization (0 p 1) Meng Wang, Student Member, IEEE, Weiyu Xu, and Ao Tang, Senior
More informationAn Introduction to Sparse Recovery (and Compressed Sensing)
An Introduction to Sparse Recovery (and Compressed Sensing) Massimo Fornasier Department of Mathematics massimo.fornasier@ma.tum.de http://www-m15.ma.tum.de/ CoSeRa 2016 RWTH - Aachen September 19, 2016
More informationLow-rank Tensor Recovery
Low-rank Tensor Recovery Dissertation zur Erlangung des Doktorgrades Dr. rer. nat. der Mathematisch-Naturwissenschaftlichen Fakultät der Rheinischen Friedrich-Wilhelms-Universität Bonn vorgelegt von Željka
More informationFast Reconstruction Algorithms for Deterministic Sensing Matrices and Applications
Fast Reconstruction Algorithms for Deterministic Sensing Matrices and Applications Program in Applied and Computational Mathematics Princeton University NJ 08544, USA. Introduction What is Compressive
More informationEE 381V: Large Scale Optimization Fall Lecture 24 April 11
EE 381V: Large Scale Optimization Fall 2012 Lecture 24 April 11 Lecturer: Caramanis & Sanghavi Scribe: Tao Huang 24.1 Review In past classes, we studied the problem of sparsity. Sparsity problem is that
More informationPre-weighted Matching Pursuit Algorithms for Sparse Recovery
Journal of Information & Computational Science 11:9 (214) 2933 2939 June 1, 214 Available at http://www.joics.com Pre-weighted Matching Pursuit Algorithms for Sparse Recovery Jingfei He, Guiling Sun, Jie
More informationInterpolation via weighted l 1 minimization
Interpolation via weighted l 1 minimization Rachel Ward University of Texas at Austin December 12, 2014 Joint work with Holger Rauhut (Aachen University) Function interpolation Given a function f : D C
More informationJointly Low-Rank and Bisparse Recovery: Questions and Partial Answers
Jointly Low-Rank and Bisparse Recovery: Questions and Partial Answers Simon Foucart, Rémi Gribonval, Laurent Jacques, and Holger Rauhut Abstract This preprint is not a finished product. It is presently
More informationSparse and Low Rank Recovery via Null Space Properties
Sparse and Low Rank Recovery via Null Space Properties Holger Rauhut Lehrstuhl C für Mathematik (Analysis), RWTH Aachen Convexity, probability and discrete structures, a geometric viewpoint Marne-la-Vallée,
More informationSparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images
Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images Alfredo Nava-Tudela ant@umd.edu John J. Benedetto Department of Mathematics jjb@umd.edu Abstract In this project we are
More informationModel-Based Compressive Sensing for Signal Ensembles. Marco F. Duarte Volkan Cevher Richard G. Baraniuk
Model-Based Compressive Sensing for Signal Ensembles Marco F. Duarte Volkan Cevher Richard G. Baraniuk Concise Signal Structure Sparse signal: only K out of N coordinates nonzero model: union of K-dimensional
More informationINDUSTRIAL MATHEMATICS INSTITUTE. B.S. Kashin and V.N. Temlyakov. IMI Preprint Series. Department of Mathematics University of South Carolina
INDUSTRIAL MATHEMATICS INSTITUTE 2007:08 A remark on compressed sensing B.S. Kashin and V.N. Temlyakov IMI Preprint Series Department of Mathematics University of South Carolina A remark on compressed
More informationECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis Lecture 3: Sparse signal recovery: A RIPless analysis of l 1 minimization Yuejie Chi The Ohio State University Page 1 Outline
More informationAnalysis of Greedy Algorithms
Analysis of Greedy Algorithms Jiahui Shen Florida State University Oct.26th Outline Introduction Regularity condition Analysis on orthogonal matching pursuit Analysis on forward-backward greedy algorithm
More informationCompressed Sensing: a Subgradient Descent Method for Missing Data Problems
Compressed Sensing: a Subgradient Descent Method for Missing Data Problems ANZIAM, Jan 30 Feb 3, 2011 Jonathan M. Borwein Jointly with D. Russell Luke, University of Goettingen FRSC FAAAS FBAS FAA Director,
More informationSimultaneous Sparsity
Simultaneous Sparsity Joel A. Tropp Anna C. Gilbert Martin J. Strauss {jtropp annacg martinjs}@umich.edu Department of Mathematics The University of Michigan 1 Simple Sparse Approximation Work in the d-dimensional,
More informationSubset selection with sparse matrices
Subset selection with sparse matrices Alberto Del Pia, University of Wisconsin-Madison Santanu S. Dey, Georgia Tech Robert Weismantel, ETH Zürich February 1, 018 Schloss Dagstuhl Subset selection for regression
More informationLeast Sparsity of p-norm based Optimization Problems with p > 1
Least Sparsity of p-norm based Optimization Problems with p > Jinglai Shen and Seyedahmad Mousavi Original version: July, 07; Revision: February, 08 Abstract Motivated by l p -optimization arising from
More informationRandomness-in-Structured Ensembles for Compressed Sensing of Images
Randomness-in-Structured Ensembles for Compressed Sensing of Images Abdolreza Abdolhosseini Moghadam Dep. of Electrical and Computer Engineering Michigan State University Email: abdolhos@msu.edu Hayder
More informationCombining geometry and combinatorics
Combining geometry and combinatorics A unified approach to sparse signal recovery Anna C. Gilbert University of Michigan joint work with R. Berinde (MIT), P. Indyk (MIT), H. Karloff (AT&T), M. Strauss
More informationMismatch and Resolution in CS
Mismatch and Resolution in CS Albert Fannjiang, UC Davis Coworker: Wenjing Liao, UC Davis SPIE Optics + Photonics, San Diego, August 2-25, 20 Mismatch: gridding error Band exclusion Local optimization
More informationBayesian Methods for Sparse Signal Recovery
Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Jason Palmer, Zhilin Zhang and Ritwik Giri Motivation Motivation Sparse Signal Recovery
More informationGeneralized greedy algorithms.
Generalized greedy algorithms. François-Xavier Dupé & Sandrine Anthoine LIF & I2M Aix-Marseille Université - CNRS - Ecole Centrale Marseille, Marseille ANR Greta Séminaire Parisien des Mathématiques Appliquées
More informationCompressed sensing and imaging
Compressed sensing and imaging The effect and benefits of local structure Ben Adcock Department of Mathematics Simon Fraser University 1 / 45 Overview Previously: An introduction to compressed sensing.
More informationContents. 0.1 Notation... 3
Contents 0.1 Notation........................................ 3 1 A Short Course on Frame Theory 4 1.1 Examples of Signal Expansions............................ 4 1.2 Signal Expansions in Finite-Dimensional
More informationNoisy Signal Recovery via Iterative Reweighted L1-Minimization
Noisy Signal Recovery via Iterative Reweighted L1-Minimization Deanna Needell UC Davis / Stanford University Asilomar SSC, November 2009 Problem Background Setup 1 Suppose x is an unknown signal in R d.
More informationSparse Optimization Lecture: Dual Certificate in l 1 Minimization
Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Instructor: Wotao Yin July 2013 Note scriber: Zheng Sun Those who complete this lecture will know what is a dual certificate for l 1 minimization
More informationSparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images!
Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images! Alfredo Nava-Tudela John J. Benedetto, advisor 1 Happy birthday Lucía! 2 Outline - Problem: Find sparse solutions
More informationNumerical Methods for Sparse Recovery
Radon Series Comp. Appl. Math xx, 0 c de Gruyter 00 Numerical Methods for Sparse Recovery Massimo Fornasier Abstract. These lecture notes address the analysis of numerical methods for performing optimizations
More informationDefinition 1. A set V is a vector space over the scalar field F {R, C} iff. there are two operations defined on V, called vector addition
6 Vector Spaces with Inned Product Basis and Dimension Section Objective(s): Vector Spaces and Subspaces Linear (In)dependence Basis and Dimension Inner Product 6 Vector Spaces and Subspaces Definition
More informationSparse representation classification and positive L1 minimization
Sparse representation classification and positive L1 minimization Cencheng Shen Joint Work with Li Chen, Carey E. Priebe Applied Mathematics and Statistics Johns Hopkins University, August 5, 2014 Cencheng
More information