Interpolation via weighted l 1 minimization
|
|
- Flora Flynn
- 5 years ago
- Views:
Transcription
1 Interpolation via weighted l 1 minimization Rachel Ward University of Texas at Austin December 12, 2014 Joint work with Holger Rauhut (Aachen University)
2 Function interpolation Given a function f : D C on a domain D, interpolate or approximate f from sample values y 1 = f (u 1 ),..., y m = f (u m ).
3 Function interpolation Given a function f : D C on a domain D, interpolate or approximate f from sample values y 1 = f (u 1 ),..., y m = f (u m ). Assume the form f (u) = j Γ x j ψ j (u), Γ = N Approaches: Standard interpolation: Choose m = N. Find appropriate Γ and sampling points u 1, u 2,..., u m. Least squares regression: Choose N < m; minimize y Ax 2 where A l,j = ψ j (u l ) is the sampling matrix. Compressive sensing methods: Choose N > m. Exploit approximate sparsity of coefficient vector x to solve the underdetermined system y = Ax.
4 Orthonormal systems L 2-normalized Legendre polynomials D: domain endowed with a probability measure ν. ψ j : D C, j Γ (finite or infinite) {ψ j } j Γ is an orthonormal system: D ψ j(t)ψ k (t)dν(t) = δ j,k
5 Orthonormal systems L 2-normalized Legendre polynomials D: domain endowed with a probability measure ν. ψ j : D C, j Γ (finite or infinite) {ψ j } j Γ is an orthonormal system: D ψ j(t)ψ k (t)dν(t) = δ j,k Examples: Trigonometric system: ψ j (t) = e 2πijt, D = [0, 1], dν(t) = dt, ψ j 1.
6 Orthonormal systems L 2-normalized Legendre polynomials D: domain endowed with a probability measure ν. ψ j : D C, j Γ (finite or infinite) {ψ j } j Γ is an orthonormal system: D ψ j(t)ψ k (t)dν(t) = δ j,k Examples: Trigonometric system: ψ j (t) = e 2πijt, D = [0, 1], dν(t) = dt, ψ j 1. L 2 -normalized Legendre polynomials L j : D = [ 1, 1], dν(t) = dt, L j = 2j + 1.
7 Orthonormal systems L 2-normalized Legendre polynomials D: domain endowed with a probability measure ν. ψ j : D C, j Γ (finite or infinite) {ψ j } j Γ is an orthonormal system: D ψ j(t)ψ k (t)dν(t) = δ j,k Tensor products of Legendre polynomials: D = [ 1, 1] d, j = (j 1, j 2,..., j d ), L j = d k=1 L j k, L j = d k=1 2jk + 1. In high-dimensional problems, smoothness is not enough to avoid curse of dimension too local! We will combine smoothness and sparsity.
8 Smoothness and weights In general, f L + f L promotes smoothness Consider ψ j (t) = e 2πijt, j Z, t [0, 1] Derivatives satisfy ψ j = 2π j, j Z. For f (t) = j x jψ j (t), f L + f L = j x j ψ j L + j x j ψ j x j ( ψ j L + ψ j L ) j Z = x j (1 + 2π j ) =: x ω,1. j Z
9 Smoothness and weights In general, f L + f L promotes smoothness Consider ψ j (t) = e 2πijt, j Z, t [0, 1] Derivatives satisfy ψ j = 2π j, j Z. For f (t) = j x jψ j (t), f L + f L = j x j ψ j L + j x j ψ j x j ( ψ j L + ψ j L ) j Z = x j (1 + 2π j ) =: x ω,1. j Z Weighted l 1 -coefficient norm promotes smoothness. It also promotes sparsity!
10 Weighted norms and weighted sparsity Weighted l 1 norm: pay a higher price to include certain indices x ω,1 = j Γ ω j x j New: weighted sparsity: x ω,0 := j:x j 0 ω 2 j x is called weighted s-sparse if x ω,0 s. Weighted best s-term approximation error: σ s (x) ω,1 := inf z: z ω,0 s x z ω,1
11 (Weighted) Compressive Sensing Recover a weighted s-sparse (or weighted-compressible) vector x from measurements y = Ax, where A C m N with m < N. Weighted l 1 -minimization min z C N z ω,1 subject to Az = y Noisy version min z C N z ω,1 subject to Az y 2 η
12 (Weighted) Compressive Sensing Recover a weighted s-sparse (or weighted-compressible) vector x from measurements y = Ax, where A C m N with m < N. Weighted l 1 -minimization min z C N z ω,1 subject to Az = y Noisy version min z C N z ω,1 subject to Az y 2 η Keep in mind sampling matrix: ψ 1 (u 1 ) ψ 2 (u 1 ) ψ N (u 1 ) ψ 1 (u 2 ) ψ 2 (u 2 ) ψ N (u 2 ) A =... ψ 1 (u m ) ψ 2 (u m ) ψ N (u m )
13 Numerical example - comparing different reconstructions 1 Original function Least squares D=15 1 Exact interpolation, D= Unweighted l1 minimizer 1 Weighted l1 minimizer, j =j 1/ f (u) = 1. Draw m = 30 sampling points u 1+25u 2 l i.i.d. from uniform measure on [ 1, 1]. Interpolate (exactly or approximately) the samples y l = f (u l ) by various choices of {x j } in f # (u) = 80 j=0 x j e πiju *Stability of unweighted l 1 minimization given exact sparsity: Rauhut, W. 09 *Stability of least squares regression: Cohen, Davenport, Leviatan 2011
14 Numerical example - comparing different reconstructions Different trials correspond to different random draws of m = 30 sampling points from uniform measure on [ 1, 1].
15 Numerical example - comparing different reconstructions Same experiment, but now interpolating/approximating by Legendre polynomials, and sampling points are from Chebyshev measure on [ 1, 1], dν(u) = du π 1 u 2.
16 What is going on? Runge s function and its Legendre polynomial coefficients Compare coefficient indices picked up by various reconstruction methods: Least squares unweighted l 1 minimization weighted l 1 minimization
17 Back to Compressive Sensing setting Recover a weighted s-sparse (or weighted-compressible) vector x from measurements y = Ax, where A C m N with m < N. Weighted l 1 -minimization min z C N N ω j z j j=1 subject to Az = y Keep in mind sampling matrix: ψ 1 (u 1 ) ψ 2 (u 1 ) ψ N (u 1 ) ψ 1 (u 2 ) ψ 2 (u 2 ) ψ N (u 2 ) A =... ψ 1 (u m ) ψ 2 (u m ) ψ N (u m )
18 Weighted restricted isometry property (WRIP) Definition (with H. Rauhut 13) The weighted restricted isometry constant δ ω,s of a matrix A C m N is defined to be the smallest constant such that (1 δ ω,s ) x 2 2 Ax 2 2 (1 + δ ω,s ) x 2 2 for all x C N with x ω,0 = l:x l 0 ω2 j s.
19 Weighted restricted isometry property (WRIP) Definition (with H. Rauhut 13) The weighted restricted isometry constant δ ω,s of a matrix A C m N is defined to be the smallest constant such that (1 δ ω,s ) x 2 2 Ax 2 2 (1 + δ ω,s ) x 2 2 for all x C N with x ω,0 = l:x l 0 ω2 j s. Unweighted uniform restricted isometry property (Candès, Tao 05): ω 1. Since we assume ω j 1, WRIP is weaker than uniform RIP Related: Model-based compressive sensing (Baraniuk, Cevher, Duarte, Hedge, Wakin 10)
20 Weighted restricted isometry property (WRIP) Definition (with H. Rauhut 13) The weighted restricted isometry constant δ ω,s of a matrix A C m N is defined to be the smallest constant such that (1 δ ω,s ) x 2 2 Ax 2 2 (1 + δ ω,s ) x 2 2 for all x C N with x ω,0 = l:x l 0 ω2 j s. Weights allow us to analyze sampling matrices from function systems with unbounded norm. Uniform RIP does not hold for such matrices!
21 Weighted RIP of random sampling matrix ψ j : D C, orthonormal system w.r.t. prob. measure ν. and ψ j ω j. Sampling points u 1,..., u m taken i.i.d. at random according to ν. Random sampling matrix A C m N with entries A lj = ψ j (u l ).
22 Weighted RIP of random sampling matrix ψ j : D C, orthonormal system w.r.t. prob. measure ν. and ψ j ω j. Sampling points u 1,..., u m taken i.i.d. at random according to ν. Random sampling matrix A C m N with entries A lj = ψ j (u l ). Fix s, and choose N so that ω 1, ω 2,..., ω N s/2. Theorem (with H. Rauhut, 13) If m Cδ 2 s log 3 (s) log(n) then the weighted restricted isometry constant of 1 m A satisfies δ ω,s δ with high probability.
23 Weighted RIP of random sampling matrix ψ j : D C, orthonormal system w.r.t. prob. measure ν. and ψ j ω j. Sampling points u 1,..., u m taken i.i.d. at random according to ν. Random sampling matrix A C m N with entries A lj = ψ j (u l ). Fix s, and choose N so that ω 1, ω 2,..., ω N s/2. Theorem (with H. Rauhut, 13) If m Cδ 2 s log 3 (s) log(n) then the weighted restricted isometry constant of 1 m A satisfies δ ω,s δ with high probability. Generalizes previous results (Candès, Tao, Rudelson, Vershynin) for uniformly bounded systems, ψ j K for all j.
24 Recovery via weighted l 1 -minimization Theorem (with H. Rauhut) Let A C m N have the WRIP with weights (ω) and δ ω,3s < 1/3. For x C N and y = Ax + e with e 2 η let x be a minimizer of Then min z ω,1 subject to Az y 2 η. x x ω,1 C 1 σ s (x) ω,1 + D 1 sη Generalizes unweighted l 1 minimization results (Candès, Romberg, Tao 06, Donoho 06).
25 Recovery via weighted l 1 -minimization Theorem (with H. Rauhut) Let A C m N have the WRIP with weights (ω) and δ ω,3s < 1/3. For x C N and y = Ax + e with e 2 η let x be a minimizer of Then min z ω,1 subject to Az y 2 η. x x ω,1 C 1 σ s (x) ω,1 + D 1 sη Generalizes unweighted l 1 minimization results (Candès, Romberg, Tao 06, Donoho 06). Should be of independent interest in structured sparse recovery problems / sparse recovery problems with nonuniform priors on support
26 Prior work on weighted l 1 minimization Weighted l 1 norm: x ω,1 = j Γ ω j x j Many previous works on the analysis of weighted l 1 minimization! Focused on the finite-dimensional setting, and with analysis based on unweighted sparsity von Borries, Miosso, and Potes 2007 Vaswani and Lu 2009, Jacques 2010, Xu 2010 Khajehnejad, Xu, Avestimehr, and Hassibi (2009 & 2010) Friedlander, Mansour, Saab, and Yilmaz 2012, Misra and Parrilo 2013 Peng, Hampton, Doostan sparse polynomial chaos approximation of high-dimensional stochastic functions
27 Back to function interpolation Suppose that Γ = N <. Given samples y 1 = f (u 1 ),..., y m = f (u m ) of f (u) = j Γ x jψ j (u) reconstruction amounts to solving y = Ax where A C m N is the sampling matrix A l,j = ψ j (u l )
28 Back to function interpolation Suppose that Γ = N <. Given samples y 1 = f (u 1 ),..., y m = f (u m ) of f (u) = j Γ x jψ j (u) reconstruction amounts to solving y = Ax where A C m N is the sampling matrix A l,j = ψ j (u l ) Previous analysis suggests: use weighted l 1 -minimization with weights ω j ψ j L or ω j ψ j L to recover weighted-sparse or weighted-compressible x when m < N. Choose u 1,..., u m i.i.d. at random according to ν ψ in order to analyze WRIP of sampling matrix. x will not be exactly sparse. Measure residual error f f # in which norm? Recall g L + g L g ω,1 if ω j ψ j L
29 Interpolation via weighted l 1 minimization Suppose f (u) = j Γ x jψ j (u), Γ = N < Set weights ω j > ψ j
30 Interpolation via weighted l 1 minimization Suppose f (u) = j Γ x jψ j (u), Γ = N < Set weights ω j > ψ j Theorem From m s log 3 (s) log(n) samples y 1 = f (u 1 ),..., y m = f (u m ) where u j ν ψ, if x is the solution to min z C Γ z ω,1 subject to Az = y and f (u) = j Γ x j ψ j(u),
31 Interpolation via weighted l 1 minimization Suppose f (u) = j Γ x jψ j (u), Γ = N < Set weights ω j > ψ j Theorem From m s log 3 (s) log(n) samples y 1 = f (u 1 ),..., y m = f (u m ) where u j ν ψ, if x is the solution to min z C Γ z ω,1 subject to Az = y and f (u) = j Γ x j ψ j(u), then with high probability, f f f f ω,1 C 1 σ(f ) ω,1
32 Interpolation via weighted l 1 minimization Suppose f (u) = j Γ x jψ j (u), Γ = N < Set weights ω j > ψ j Theorem From m s log 3 (s) log(n) samples y 1 = f (u 1 ),..., y m = f (u m ) where u j ν ψ, if x is the solution to min z C Γ z ω,1 subject to Az = y and f (u) = j Γ x j ψ j(u), then with high probability, f f f f ω,1 C 1 σ(f ) ω,1 More realistic: Γ =. How to pass to a finite dimensional approximation in a principled way?
33 Weighted function spaces A ω,p = {f : f (u) = j Γ x j ψ j (u), f ω,p := x ω,p < } Interesting range: 0 p 1 A ω,p A ω,q if p < q Solving m s log 3 (s) log(n) for s, a Stechkin-type error bound yields ( ) log(n) f f L f f 4 1/p 1 ω,1 C 1 f ω,p m
34 Approximation in infinite-dimensional spaces Suppose Γ =, lim j ω j = and ω j > ψ j.
35 Approximation in infinite-dimensional spaces Suppose Γ =, lim j ω j = and ω j > ψ j. Theorem Suppose f A ω,p for some 0 < p < 1. Fix s, and set Γ s = {j Γ : ωj 2 s/2}.
36 Approximation in infinite-dimensional spaces Suppose Γ =, lim j ω j = and ω j > ψ j. Theorem Suppose f A ω,p for some 0 < p < 1. Fix s, and set Γ s = {j Γ : ωj 2 s/2}. Draw m Cs log 3 (s) log( Γ s ) samples y 1 = f (u 1 ),..., y m = f (u m ) according to ν ψ, and let x be the solution to min z C Γs z ω,1 subject to Ax y 2 m/s f f Γs ω,1 Put f = j Γ s x j ψ j.
37 Approximation in infinite-dimensional spaces Suppose Γ =, lim j ω j = and ω j > ψ j. Theorem Suppose f A ω,p for some 0 < p < 1. Fix s, and set Γ s = {j Γ : ωj 2 s/2}. Draw m Cs log 3 (s) log( Γ s ) samples y 1 = f (u 1 ),..., y m = f (u m ) according to ν ψ, and let x be the solution to min z C Γs z ω,1 subject to Ax y 2 m/s f f Γs ω,1 Put f = j Γ s x j ψ j. Then with high probability, ( log f f L f f 4 ) 1/p 1 (N) ω,1 C 1 f ω,p m
38 Approximation in infinite-dimensional spaces Suppose Γ =, lim j ω j = and ω j > ψ j. Theorem Suppose f A ω,p for some 0 < p < 1. Fix s, and set Γ s = {j Γ : ωj 2 s/2}. Draw m Cs log 3 (s) log( Γ s ) samples y 1 = f (u 1 ),..., y m = f (u m ) according to ν ψ, and let x be the solution to min z C Γs z ω,1 subject to Ax y 2 m/s f f Γs ω,1 Put f = j Γ s x j ψ j. Then with high probability, ( log f f L f f 4 ) 1/p 1 (N) ω,1 C 1 f ω,p m **Using greedy alternative such as weighted iterative hard thresholding, do not need to know m/s f f Γs ω,1, get same bound
39 Function approximation in high dimensions Important: For domains D = [ 1, 1] d, don t want number of measurements m in resulting bound to grow much with d.
40 Function approximation in high dimensions Important: For domains D = [ 1, 1] d, don t want number of measurements m in resulting bound to grow much with d. L 2 -normalized Chebyshev polynomials T j (u) = 2 cos(j arccos u), T 0 (u) 1, j N, Tensorized Chebyshev polynomials T k (u) = d T kj (u j ) = j=1 j supp k T kj (u j ), u = (u j ) j 1, k F, F = {k = (k 1, k 2,...), k j N}}
41 Function approximation in high dimensions Important: For domains D = [ 1, 1] d, don t want number of measurements m in resulting bound to grow much with d. L 2 -normalized Chebyshev polynomials T j (u) = 2 cos(j arccos u), T 0 (u) 1, j N, Tensorized Chebyshev polynomials T k (u) = d T kj (u j ) = j=1 Product probability measure j supp k T kj (u j ), u = (u j ) j 1, k F, F = {k = (k 1, k 2,...), k j N}} η = j 1 du j π 1 uj 2, D. {T k } k F forms orthonormal basis for L 2 (D, dη).
42 Sparse recovery for tensorized Chebyshev polynomials L -bound for the tensorized Chebyshev polynomials T k : Choice of weights: T k = 2 k 0/2. ω k = d (k j + 1) 1/2 2 k 0/2 j=1 Encourages both sparse and low-order tensor products
43 Sparse recovery for tensorized Chebyshev polynomials L -bound for the tensorized Chebyshev polynomials T k : Choice of weights: T k = 2 k 0/2. ω k = d (k j + 1) 1/2 2 k 0/2 j=1 Encourages both sparse and low-order tensor products Subset of indices forms a hyperbolic cross: Λ 0 = {k N d 0 : ω 2 k s} [Kuhn, Sickel, Ulrich, 2014, Cohen, DeVore, Foucart, Rauhut 2011]: Λ 0 e d s 2+log(d)
44 Apply weighted l 1 theory: Consider a function f = k Λ x k T k on [ 1, 1] d, f A ω,p Fix s, and fix Λ 0 = {k N d : ωk 2 s} Reduced basis Fix number of samples m Cs log 4 (s) log(d) Draw m samples y l = f (u l ) according to Chebyshev measure Let A C m N be sampling matrix with entries A l,k = T k (u l ). Let x # be solution of min x ω,1 subject to Ax y 2 m/s f f Λ0 ω,1 and set f # = k Λ 0 x # k T k.
45 Apply weighted l 1 theory: Consider a function f = k Λ x k T k on [ 1, 1] d, f A ω,p Fix s, and fix Λ 0 = {k N d : ωk 2 s} Reduced basis Fix number of samples m Cs log 4 (s) log(d) Draw m samples y l = f (u l ) according to Chebyshev measure Let A C m N be sampling matrix with entries A l,k = T k (u l ). Let x # be solution of min x ω,1 subject to Ax y 2 m/s f f Λ0 ω,1 and set f # = k Λ 0 x # k T k. Then with probability exceeding 1 e d log3 (s), ( log f f 4 ) 1/p 1 (s) log(d) L C 1 f ω,p m
46 Limitations of weighted l 1 approach In the previous example, N = Λ 0 = e d s 2+log 2 (d) Weighted l 1 minimization as a reconstruction method on such large scale problems is impractical Fix s s. Least squares projection onto is faster, but too greedy Can we meet in the middle? Ñ = e d s 2+log 2 (d)
47 Summary We introduced weighted l 1 minimization for stable and robust function interpolation, as taking into account both sparsity and smoothness present in natural functions of interest Along the way, we extended the notion of restricted isometry property to weighted restricted isometry property, a more mild condition that allows us to treat function systems with increasing norm Weighted l 1 minimization can overcome curse of dimension w.r.t. number of samples in high-dimensional approximation problems.
48 Extensions: We observe empirically that weighted l 1 minimization is faster than unweighted l 1 minimization. The steeper the weights, the faster. Justificaton? We observe that, with error on measurements y = f (u) + ξ, reconstruction results are similar, provided that the regularization parameter is chosen correctly in regularization methods. Equality constrained l 1 minimization, weighted or not, leads to overfitting. Why?
49 References Rauhut, Ward. Interpolation via weighted l 1 minimization. Arxiv preprint, Rauhut, Ward, Sparse Legendre expansions via l 1 -minimization. Journal of approximation theory (2012): Cohen, Davenport, Leviatan. On the stability and accuracy of least squares approximations. Foundations of computational mathematics, 13 (2013), Jo. Iterative Hard Thresholding for weighted sparse approximation. Arxiv preprint, Burq, Dyatlov, Ward, Zworski. Weighted eigenfunction estimates with applications to compressed sensing. SIAM J. Math. Anal. 44(5) (2012)
Interpolation via weighted l 1 -minimization
Interpolation via weighted l 1 -minimization Holger Rauhut RWTH Aachen University Lehrstuhl C für Mathematik (Analysis) Matheon Workshop Compressive Sensing and Its Applications TU Berlin, December 11,
More informationInterpolation via weighted l 1 -minimization
Interpolation via weighted l 1 -minimization Holger Rauhut RWTH Aachen University Lehrstuhl C für Mathematik (Analysis) Mathematical Analysis and Applications Workshop in honor of Rupert Lasser Helmholtz
More informationSparse Legendre expansions via l 1 minimization
Sparse Legendre expansions via l 1 minimization Rachel Ward, Courant Institute, NYU Joint work with Holger Rauhut, Hausdorff Center for Mathematics, Bonn, Germany. June 8, 2010 Outline Sparse recovery
More informationInterpolation via weighted l 1 minimization
Interpolation via weighted l minimization Holger Rauhut, Rachel Ward August 3, 23 Abstract Functions of interest are often smooth and sparse in some sense, and both priors should be taken into account
More informationStrengthened Sobolev inequalities for a random subspace of functions
Strengthened Sobolev inequalities for a random subspace of functions Rachel Ward University of Texas at Austin April 2013 2 Discrete Sobolev inequalities Proposition (Sobolev inequality for discrete images)
More informationRecovering overcomplete sparse representations from structured sensing
Recovering overcomplete sparse representations from structured sensing Deanna Needell Claremont McKenna College Feb. 2015 Support: Alfred P. Sloan Foundation and NSF CAREER #1348721. Joint work with Felix
More informationAN INTRODUCTION TO COMPRESSIVE SENSING
AN INTRODUCTION TO COMPRESSIVE SENSING Rodrigo B. Platte School of Mathematical and Statistical Sciences APM/EEE598 Reverse Engineering of Complex Dynamical Networks OUTLINE 1 INTRODUCTION 2 INCOHERENCE
More informationSparse recovery for spherical harmonic expansions
Rachel Ward 1 1 Courant Institute, New York University Workshop Sparsity and Cosmology, Nice May 31, 2011 Cosmic Microwave Background Radiation (CMB) map Temperature is measured as T (θ, ϕ) = k k=0 l=
More informationExact Reconstruction Conditions and Error Bounds for Regularized Modified Basis Pursuit (Reg-Modified-BP)
1 Exact Reconstruction Conditions and Error Bounds for Regularized Modified Basis Pursuit (Reg-Modified-BP) Wei Lu and Namrata Vaswani Department of Electrical and Computer Engineering, Iowa State University,
More informationExponential decay of reconstruction error from binary measurements of sparse signals
Exponential decay of reconstruction error from binary measurements of sparse signals Deanna Needell Joint work with R. Baraniuk, S. Foucart, Y. Plan, and M. Wootters Outline Introduction Mathematical Formulation
More informationCompressed Sensing and Sparse Recovery
ELE 538B: Sparsity, Structure and Inference Compressed Sensing and Sparse Recovery Yuxin Chen Princeton University, Spring 217 Outline Restricted isometry property (RIP) A RIPless theory Compressed sensing
More informationNear Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing
Near Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas M.Vidyasagar@utdallas.edu www.utdallas.edu/ m.vidyasagar
More informationModel-Based Compressive Sensing for Signal Ensembles. Marco F. Duarte Volkan Cevher Richard G. Baraniuk
Model-Based Compressive Sensing for Signal Ensembles Marco F. Duarte Volkan Cevher Richard G. Baraniuk Concise Signal Structure Sparse signal: only K out of N coordinates nonzero model: union of K-dimensional
More informationInterpolation-Based Trust-Region Methods for DFO
Interpolation-Based Trust-Region Methods for DFO Luis Nunes Vicente University of Coimbra (joint work with A. Bandeira, A. R. Conn, S. Gratton, and K. Scheinberg) July 27, 2010 ICCOPT, Santiago http//www.mat.uc.pt/~lnv
More informationSparse and Low Rank Recovery via Null Space Properties
Sparse and Low Rank Recovery via Null Space Properties Holger Rauhut Lehrstuhl C für Mathematik (Analysis), RWTH Aachen Convexity, probability and discrete structures, a geometric viewpoint Marne-la-Vallée,
More informationLecture Notes 9: Constrained Optimization
Optimization-based data analysis Fall 017 Lecture Notes 9: Constrained Optimization 1 Compressed sensing 1.1 Underdetermined linear inverse problems Linear inverse problems model measurements of the form
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationGreedy Signal Recovery and Uniform Uncertainty Principles
Greedy Signal Recovery and Uniform Uncertainty Principles SPIE - IE 2008 Deanna Needell Joint work with Roman Vershynin UC Davis, January 2008 Greedy Signal Recovery and Uniform Uncertainty Principles
More informationStability and Robustness of Weak Orthogonal Matching Pursuits
Stability and Robustness of Weak Orthogonal Matching Pursuits Simon Foucart, Drexel University Abstract A recent result establishing, under restricted isometry conditions, the success of sparse recovery
More informationNoisy Signal Recovery via Iterative Reweighted L1-Minimization
Noisy Signal Recovery via Iterative Reweighted L1-Minimization Deanna Needell UC Davis / Stanford University Asilomar SSC, November 2009 Problem Background Setup 1 Suppose x is an unknown signal in R d.
More informationCOMPRESSIVE SENSING PETROV-GALERKIN APPROXIMATION OF HIGH-DIMENSIONAL PARAMETRIC OPERATOR EQUATIONS
COMPRESSIVE SENSING PETROV-GALERKIN APPROXIMATION OF HIGH-DIMENSIONAL PARAMETRIC OPERATOR EQUATIONS HOLGER RAUHUT AND CHRISTOPH SCHWAB Abstract. We analyze the convergence of compressive sensing based
More informationCoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles
CoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles SIAM Student Research Conference Deanna Needell Joint work with Roman Vershynin and Joel Tropp UC Davis, May 2008 CoSaMP: Greedy Signal
More informationRecent Developments in Compressed Sensing
Recent Developments in Compressed Sensing M. Vidyasagar Distinguished Professor, IIT Hyderabad m.vidyasagar@iith.ac.in, www.iith.ac.in/ m vidyasagar/ ISL Seminar, Stanford University, 19 April 2018 Outline
More informationGREEDY SIGNAL RECOVERY REVIEW
GREEDY SIGNAL RECOVERY REVIEW DEANNA NEEDELL, JOEL A. TROPP, ROMAN VERSHYNIN Abstract. The two major approaches to sparse recovery are L 1-minimization and greedy methods. Recently, Needell and Vershynin
More informationCompressed Sensing and Redundant Dictionaries
Compressed Sensing and Redundant Dictionaries Holger Rauhut, Karin Schnass and Pierre Vandergheynst Abstract This article extends the concept of compressed sensing to signals that are not sparse in an
More informationSparse Approximation of PDEs based on Compressed Sensing
Sparse Approximation of PDEs based on Compressed Sensing Simone Brugiapaglia Department of Mathematics Simon Fraser University Retreat for Young Researchers in Stochastics September 24, 26 2 Introduction
More informationSimultaneous Sparsity
Simultaneous Sparsity Joel A. Tropp Anna C. Gilbert Martin J. Strauss {jtropp annacg martinjs}@umich.edu Department of Mathematics The University of Michigan 1 Simple Sparse Approximation Work in the d-dimensional,
More informationCoSaMP. Iterative signal recovery from incomplete and inaccurate samples. Joel A. Tropp
CoSaMP Iterative signal recovery from incomplete and inaccurate samples Joel A. Tropp Applied & Computational Mathematics California Institute of Technology jtropp@acm.caltech.edu Joint with D. Needell
More informationRui ZHANG Song LI. Department of Mathematics, Zhejiang University, Hangzhou , P. R. China
Acta Mathematica Sinica, English Series May, 015, Vol. 31, No. 5, pp. 755 766 Published online: April 15, 015 DOI: 10.1007/s10114-015-434-4 Http://www.ActaMath.com Acta Mathematica Sinica, English Series
More informationCompressed Sensing and Redundant Dictionaries
Compressed Sensing and Redundant Dictionaries Holger Rauhut, Karin Schnass and Pierre Vandergheynst December 2, 2006 Abstract This article extends the concept of compressed sensing to signals that are
More informationCompressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery
Compressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery Jorge F. Silva and Eduardo Pavez Department of Electrical Engineering Information and Decision Systems Group Universidad
More informationCOMPRESSED SENSING IN PYTHON
COMPRESSED SENSING IN PYTHON Sercan Yıldız syildiz@samsi.info February 27, 2017 OUTLINE A BRIEF INTRODUCTION TO COMPRESSED SENSING A BRIEF INTRODUCTION TO CVXOPT EXAMPLES A Brief Introduction to Compressed
More informationReconstruction from Anisotropic Random Measurements
Reconstruction from Anisotropic Random Measurements Mark Rudelson and Shuheng Zhou The University of Michigan, Ann Arbor Coding, Complexity, and Sparsity Workshop, 013 Ann Arbor, Michigan August 7, 013
More informationThresholds for the Recovery of Sparse Solutions via L1 Minimization
Thresholds for the Recovery of Sparse Solutions via L Minimization David L. Donoho Department of Statistics Stanford University 39 Serra Mall, Sequoia Hall Stanford, CA 9435-465 Email: donoho@stanford.edu
More informationNew Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit
New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence
More informationKomprimované snímání a LASSO jako metody zpracování vysocedimenzionálních dat
Komprimované snímání a jako metody zpracování vysocedimenzionálních dat Jan Vybíral (Charles University Prague, Czech Republic) November 2014 VUT Brno 1 / 49 Definition and motivation Its use in bioinformatics
More informationOn the coherence barrier and analogue problems in compressed sensing
On the coherence barrier and analogue problems in compressed sensing Clarice Poon University of Cambridge June 1, 2017 Joint work with: Ben Adcock Anders Hansen Bogdan Roman (Simon Fraser) (Cambridge)
More informationUniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit
Uniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit arxiv:0707.4203v2 [math.na] 14 Aug 2007 Deanna Needell Department of Mathematics University of California,
More informationExact Low-rank Matrix Recovery via Nonconvex M p -Minimization
Exact Low-rank Matrix Recovery via Nonconvex M p -Minimization Lingchen Kong and Naihua Xiu Department of Applied Mathematics, Beijing Jiaotong University, Beijing, 100044, People s Republic of China E-mail:
More informationCompressed Sensing - Near Optimal Recovery of Signals from Highly Incomplete Measurements
Compressed Sensing - Near Optimal Recovery of Signals from Highly Incomplete Measurements Wolfgang Dahmen Institut für Geometrie und Praktische Mathematik RWTH Aachen and IMI, University of Columbia, SC
More informationEnhanced Compressive Sensing and More
Enhanced Compressive Sensing and More Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Nonlinear Approximation Techniques Using L1 Texas A & M University
More informationQuantifying conformation fluctuation induced uncertainty in bio-molecular systems
Quantifying conformation fluctuation induced uncertainty in bio-molecular systems Guang Lin, Dept. of Mathematics & School of Mechanical Engineering, Purdue University Collaborative work with Huan Lei,
More informationSensing systems limited by constraints: physical size, time, cost, energy
Rebecca Willett Sensing systems limited by constraints: physical size, time, cost, energy Reduce the number of measurements needed for reconstruction Higher accuracy data subject to constraints Original
More informationIntroduction How it works Theory behind Compressed Sensing. Compressed Sensing. Huichao Xue. CS3750 Fall 2011
Compressed Sensing Huichao Xue CS3750 Fall 2011 Table of Contents Introduction From News Reports Abstract Definition How it works A review of L 1 norm The Algorithm Backgrounds for underdetermined linear
More informationIEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER
IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER 2015 1239 Preconditioning for Underdetermined Linear Systems with Sparse Solutions Evaggelia Tsiligianni, StudentMember,IEEE, Lisimachos P. Kondi,
More informationRecovery of Compressible Signals in Unions of Subspaces
1 Recovery of Compressible Signals in Unions of Subspaces Marco F. Duarte, Chinmay Hegde, Volkan Cevher, and Richard G. Baraniuk Department of Electrical and Computer Engineering Rice University Abstract
More informationTractable Upper Bounds on the Restricted Isometry Constant
Tractable Upper Bounds on the Restricted Isometry Constant Alex d Aspremont, Francis Bach, Laurent El Ghaoui Princeton University, École Normale Supérieure, U.C. Berkeley. Support from NSF, DHS and Google.
More informationSignal Recovery From Incomplete and Inaccurate Measurements via Regularized Orthogonal Matching Pursuit
Signal Recovery From Incomplete and Inaccurate Measurements via Regularized Orthogonal Matching Pursuit Deanna Needell and Roman Vershynin Abstract We demonstrate a simple greedy algorithm that can reliably
More informationUniqueness Conditions for A Class of l 0 -Minimization Problems
Uniqueness Conditions for A Class of l 0 -Minimization Problems Chunlei Xu and Yun-Bin Zhao October, 03, Revised January 04 Abstract. We consider a class of l 0 -minimization problems, which is to search
More informationSparse solutions of underdetermined systems
Sparse solutions of underdetermined systems I-Liang Chern September 22, 2016 1 / 16 Outline Sparsity and Compressibility: the concept for measuring sparsity and compressibility of data Minimum measurements
More informationAn Introduction to Sparse Approximation
An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,
More informationSparse Recovery with Pre-Gaussian Random Matrices
Sparse Recovery with Pre-Gaussian Random Matrices Simon Foucart Laboratoire Jacques-Louis Lions Université Pierre et Marie Curie Paris, 75013, France Ming-Jun Lai Department of Mathematics University of
More informationof Orthogonal Matching Pursuit
A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit Qun Mo arxiv:50.0708v [cs.it] 8 Jan 205 Abstract We shall show that if the restricted isometry constant (RIC) δ s+ (A) of the measurement
More informationRandom projections. 1 Introduction. 2 Dimensionality reduction. Lecture notes 5 February 29, 2016
Lecture notes 5 February 9, 016 1 Introduction Random projections Random projections are a useful tool in the analysis and processing of high-dimensional data. We will analyze two applications that use
More informationPre-weighted Matching Pursuit Algorithms for Sparse Recovery
Journal of Information & Computational Science 11:9 (214) 2933 2939 June 1, 214 Available at http://www.joics.com Pre-weighted Matching Pursuit Algorithms for Sparse Recovery Jingfei He, Guiling Sun, Jie
More informationCombining geometry and combinatorics
Combining geometry and combinatorics A unified approach to sparse signal recovery Anna C. Gilbert University of Michigan joint work with R. Berinde (MIT), P. Indyk (MIT), H. Karloff (AT&T), M. Strauss
More informationFast Hard Thresholding with Nesterov s Gradient Method
Fast Hard Thresholding with Nesterov s Gradient Method Volkan Cevher Idiap Research Institute Ecole Polytechnique Federale de ausanne volkan.cevher@epfl.ch Sina Jafarpour Department of Computer Science
More informationCompressive Sensing and Beyond
Compressive Sensing and Beyond Sohail Bahmani Gerorgia Tech. Signal Processing Compressed Sensing Signal Models Classics: bandlimited The Sampling Theorem Any signal with bandwidth B can be recovered
More informationOn Rank Awareness, Thresholding, and MUSIC for Joint Sparse Recovery
On Rank Awareness, Thresholding, and MUSIC for Joint Sparse Recovery Jeffrey D. Blanchard a,1,, Caleb Leedy a,2, Yimin Wu a,2 a Department of Mathematics and Statistics, Grinnell College, Grinnell, IA
More informationOptimization for Compressed Sensing
Optimization for Compressed Sensing Robert J. Vanderbei 2014 March 21 Dept. of Industrial & Systems Engineering University of Florida http://www.princeton.edu/ rvdb Lasso Regression The problem is to solve
More informationStochastic geometry and random matrix theory in CS
Stochastic geometry and random matrix theory in CS IPAM: numerical methods for continuous optimization University of Edinburgh Joint with Bah, Blanchard, Cartis, and Donoho Encoder Decoder pair - Encoder/Decoder
More informationA New Estimate of Restricted Isometry Constants for Sparse Solutions
A New Estimate of Restricted Isometry Constants for Sparse Solutions Ming-Jun Lai and Louis Y. Liu January 12, 211 Abstract We show that as long as the restricted isometry constant δ 2k < 1/2, there exist
More informationORTHOGONAL matching pursuit (OMP) is the canonical
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 9, SEPTEMBER 2010 4395 Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property Mark A. Davenport, Member, IEEE, and Michael
More informationSparse Interactions: Identifying High-Dimensional Multilinear Systems via Compressed Sensing
Sparse Interactions: Identifying High-Dimensional Multilinear Systems via Compressed Sensing Bobak Nazer and Robert D. Nowak University of Wisconsin, Madison Allerton 10/01/10 Motivation: Virus-Host Interaction
More informationThe Sparsest Solution of Underdetermined Linear System by l q minimization for 0 < q 1
The Sparsest Solution of Underdetermined Linear System by l q minimization for 0 < q 1 Simon Foucart Department of Mathematics Vanderbilt University Nashville, TN 3784. Ming-Jun Lai Department of Mathematics,
More informationSparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery
Sparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery Anna C. Gilbert Department of Mathematics University of Michigan Sparse signal recovery measurements:
More informationUniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit
Claremont Colleges Scholarship @ Claremont CMC Faculty Publications and Research CMC Faculty Scholarship 6-5-2008 Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit
More informationMultiplicative and Additive Perturbation Effects on the Recovery of Sparse Signals on the Sphere using Compressed Sensing
Multiplicative and Additive Perturbation Effects on the Recovery of Sparse Signals on the Sphere using Compressed Sensing ibeltal F. Alem, Daniel H. Chae, and Rodney A. Kennedy The Australian National
More informationUncertainty quantification for sparse solutions of random PDEs
Uncertainty quantification for sparse solutions of random PDEs L. Mathelin 1 K.A. Gallivan 2 1 LIMSI - CNRS Orsay, France 2 Mathematics Dpt., Florida State University Tallahassee, FL, USA SIAM 10 July
More informationIntroduction to Compressed Sensing
Introduction to Compressed Sensing Alejandro Parada, Gonzalo Arce University of Delaware August 25, 2016 Motivation: Classical Sampling 1 Motivation: Classical Sampling Issues Some applications Radar Spectral
More informationGeneralized Orthogonal Matching Pursuit- A Review and Some
Generalized Orthogonal Matching Pursuit- A Review and Some New Results Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur, INDIA Table of Contents
More informationSparsest Solutions of Underdetermined Linear Systems via l q -minimization for 0 < q 1
Sparsest Solutions of Underdetermined Linear Systems via l q -minimization for 0 < q 1 Simon Foucart Department of Mathematics Vanderbilt University Nashville, TN 3740 Ming-Jun Lai Department of Mathematics
More informationA new method on deterministic construction of the measurement matrix in compressed sensing
A new method on deterministic construction of the measurement matrix in compressed sensing Qun Mo 1 arxiv:1503.01250v1 [cs.it] 4 Mar 2015 Abstract Construction on the measurement matrix A is a central
More informationStability and robustness of l 1 -minimizations with Weibull matrices and redundant dictionaries
Stability and robustness of l 1 -minimizations with Weibull matrices and redundant dictionaries Simon Foucart, Drexel University Abstract We investigate the recovery of almost s-sparse vectors x C N from
More information6 Compressed Sensing and Sparse Recovery
6 Compressed Sensing and Sparse Recovery Most of us have noticed how saving an image in JPEG dramatically reduces the space it occupies in our hard drives as oppose to file types that save the pixel value
More informationDimensionality Reduction Notes 3
Dimensionality Reduction Notes 3 Jelani Nelson minilek@seas.harvard.edu August 13, 2015 1 Gordon s theorem Let T be a finite subset of some normed vector space with norm X. We say that a sequence T 0 T
More informationCOMPRESSED Sensing (CS) is a method to recover a
1 Sample Complexity of Total Variation Minimization Sajad Daei, Farzan Haddadi, Arash Amini Abstract This work considers the use of Total Variation (TV) minimization in the recovery of a given gradient
More informationBreaking the coherence barrier - A new theory for compressed sensing
Breaking the coherence barrier - A new theory for compressed sensing Anders C. Hansen (Cambridge) Joint work with: B. Adcock (Simon Fraser University) C. Poon (Cambridge) B. Roman (Cambridge) UCL-Duke
More informationCompressive Sensing of Sparse Tensor
Shmuel Friedland Univ. Illinois at Chicago Matheon Workshop CSA2013 December 12, 2013 joint work with Q. Li and D. Schonfeld, UIC Abstract Conventional Compressive sensing (CS) theory relies on data representation
More informationBhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego
Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego 1 Outline Course Outline Motivation for Course Sparse Signal Recovery Problem Applications Computational
More informationBeyond incoherence and beyond sparsity: compressed sensing in the real world
Beyond incoherence and beyond sparsity: compressed sensing in the real world Clarice Poon 1st November 2013 University of Cambridge, UK Applied Functional and Harmonic Analysis Group Head of Group Anders
More informationThe Fundamentals of Compressive Sensing
The Fundamentals of Compressive Sensing Mark A. Davenport Georgia Institute of Technology School of Electrical and Computer Engineering Sensor Explosion Data Deluge Digital Revolution If we sample a signal
More informationExtracting structured dynamical systems using sparse optimization with very few samples
Extracting structured dynamical systems using sparse optimization with very few samples Hayden Schaeffer 1, Giang Tran 2, Rachel Ward 3 and Linan Zhang 1 1 Department of Mathematical Sciences, Carnegie
More informationPhase Transition Phenomenon in Sparse Approximation
Phase Transition Phenomenon in Sparse Approximation University of Utah/Edinburgh L1 Approximation: May 17 st 2008 Convex polytopes Counting faces Sparse Representations via l 1 Regularization Underdetermined
More informationSigma Delta Quantization for Compressed Sensing
Sigma Delta Quantization for Compressed Sensing C. Sinan Güntürk, 1 Mark Lammers, 2 Alex Powell, 3 Rayan Saab, 4 Özgür Yılmaz 4 1 Courant Institute of Mathematical Sciences, New York University, NY, USA.
More informationSparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery
Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan Connection between... Sparse Approximation and Compressed
More informationExact Signal Recovery from Sparsely Corrupted Measurements through the Pursuit of Justice
Exact Signal Recovery from Sparsely Corrupted Measurements through the Pursuit of Justice Jason N. Laska, Mark A. Davenport, Richard G. Baraniuk Department of Electrical and Computer Engineering Rice University
More informationThe Analysis Cosparse Model for Signals and Images
The Analysis Cosparse Model for Signals and Images Raja Giryes Computer Science Department, Technion. The research leading to these results has received funding from the European Research Council under
More informationMultipath Matching Pursuit
Multipath Matching Pursuit Submitted to IEEE trans. on Information theory Authors: S. Kwon, J. Wang, and B. Shim Presenter: Hwanchol Jang Multipath is investigated rather than a single path for a greedy
More informationLecture 22: More On Compressed Sensing
Lecture 22: More On Compressed Sensing Scribed by Eric Lee, Chengrun Yang, and Sebastian Ament Nov. 2, 207 Recap and Introduction Basis pursuit was the method of recovering the sparsest solution to an
More informationINDUSTRIAL MATHEMATICS INSTITUTE. B.S. Kashin and V.N. Temlyakov. IMI Preprint Series. Department of Mathematics University of South Carolina
INDUSTRIAL MATHEMATICS INSTITUTE 2007:08 A remark on compressed sensing B.S. Kashin and V.N. Temlyakov IMI Preprint Series Department of Mathematics University of South Carolina A remark on compressed
More informationParameterized PDEs Compressing sensing Sampling complexity lower-rip CS for PDEs Nonconvex regularizations Concluding remarks. Clayton G.
COMPUTATIONAL & APPLIED MATHEMATICS Parameterized PDEs Compressing sensing Sampling complexity lower-rip CS for PDEs Nonconvex regularizations Concluding remarks Polynomial approximations via compressed
More informationCompressed Sensing and Affine Rank Minimization Under Restricted Isometry
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 61, NO. 13, JULY 1, 2013 3279 Compressed Sensing Affine Rank Minimization Under Restricted Isometry T. Tony Cai Anru Zhang Abstract This paper establishes new
More informationAccelerated large-scale inversion with message passing Felix J. Herrmann, the University of British Columbia, Canada
Accelerated large-scale inversion with message passing Felix J. Herrmann, the University of British Columbia, Canada SUMMARY To meet current-day challenges, exploration seismology increasingly relies on
More informationCompressed Sensing: Lecture I. Ronald DeVore
Compressed Sensing: Lecture I Ronald DeVore Motivation Compressed Sensing is a new paradigm for signal/image/function acquisition Motivation Compressed Sensing is a new paradigm for signal/image/function
More informationOptimisation Combinatoire et Convexe.
Optimisation Combinatoire et Convexe. Low complexity models, l 1 penalties. A. d Aspremont. M1 ENS. 1/36 Today Sparsity, low complexity models. l 1 -recovery results: three approaches. Extensions: matrix
More informationTwo added structures in sparse recovery: nonnegativity and disjointedness. Simon Foucart University of Georgia
Two added structures in sparse recovery: nonnegativity and disjointedness Simon Foucart University of Georgia Semester Program on High-Dimensional Approximation ICERM 7 October 2014 Part I: Nonnegative
More informationRSP-Based Analysis for Sparsest and Least l 1 -Norm Solutions to Underdetermined Linear Systems
1 RSP-Based Analysis for Sparsest and Least l 1 -Norm Solutions to Underdetermined Linear Systems Yun-Bin Zhao IEEE member Abstract Recently, the worse-case analysis, probabilistic analysis and empirical
More informationConstructing Explicit RIP Matrices and the Square-Root Bottleneck
Constructing Explicit RIP Matrices and the Square-Root Bottleneck Ryan Cinoman July 18, 2018 Ryan Cinoman Constructing Explicit RIP Matrices July 18, 2018 1 / 36 Outline 1 Introduction 2 Restricted Isometry
More information18.S096: Compressed Sensing and Sparse Recovery
18.S096: Compressed Sensing and Sparse Recovery Topics in Mathematics of Data Science Fall 2015) Afonso S. Bandeira bandeira@mit.edu http://math.mit.edu/~bandeira November 10, 2015 These are lecture notes
More information