Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France

Size: px
Start display at page:

Download "Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France"

Transcription

1 Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr

2 Structure of the tutorial Session 1: Introduction to inverse problems & sparse models Sparse recovery: Well-posedness Complexity Pursuits Optimization principles Greedy algorithms Session 2: Recovery guarantees Coherence Restricted Isometry Property Null Space Properties 2

3 Further material on sparsity Books Signal Processing perspective S. Mallat, «Wavelet Tour of Signal Processing», 3rd edition, 2008 M. Elad, «Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing», Mathematical perspective S. Foucart, H. Rauhut, «A Mathematical Introduction to Compressed Sensing», Springer, in preparation. Review paper: Bruckstein, Donoho, Elad, SIAM Reviews,

4 Inverse problems -

5 Example: Inpainting Problem Unknown image with N pixels y 2 R N y Partially observed image: m < N observed pixels b[p] =y[p], p 2 Observed Measurement matrix b = My b y 5

6 Example : audio source separation «Softly as in a morning sunrise» 6

7 Example : audio source separation «Softly as in a morning sunrise» 6

8 Blind Source Separation Mixing model : linear instantaneous mixture y right (t) y left (t) s 1 (t) s 2 (t) s 3 (t) Source model : if disjoint time-supports y left (t) y right (t)... then clustering to : 1- identify (columns of) the mixing matrix 2- recover sources 7

9 Blind Source Separation Mixing model : linear instantaneous mixture y right (t) y left (t) s 1 (t) s 2 (t) s 3 (t) In practice... y left (t) y right (t) 8

10 Time-Frequency Masking Mixing model in the time-frequency domain Y right (, f) Y left (, f) S(, f) And miraculously... Y left (, f)... time-frequency representations of audio signals are (often) almost disjoint. Y right (, f) 9

11 Inverse problems & Sparsity -

12 Inverse problems Inverse problem : exploit indirect or incomplete obervation to recontruct some data z = My Observations Reconstruct Data Sparsity : represent / approximate high-dimensional & complex data using few parameters fewer equations than unknowns y x few nonzero components Data Reduce the dimension Representation 11

13 Inverse problems Signal space ~ R N Set of signals of interest Nonlinear Approximation = Sparse recovery Linear projection Courtesy: M. Davies, U. Edinburgh Observation space ~ R M M<<N 12

14 Sparsity: definition A vector is sparse if it has (many) zero coefficients k-sparse if it has k nonzero coefficients 13

15 Sparsity: definition Not sparse A vector is sparse if it has (many) zero coefficients k-sparse if it has k nonzero coefficients Symbolic representation as column vector 13

16 Sparsity: definition Not sparse A vector is sparse if it has (many) zero coefficients k-sparse if it has k nonzero coefficients Symbolic representation as column vector 3-sparse 13

17 Sparsity: definition Not sparse A vector is sparse if it has (many) zero coefficients k-sparse if it has k nonzero coefficients Symbolic representation as column vector Support = indices of nonzero components 3-sparse 13

18 Sparsity: definition Not sparse A vector is sparse if it has (many) zero coefficients k-sparse if it has k nonzero coefficients Symbolic representation as column vector Support = indices of nonzero components Sparsity measured with L0 pseudo-norm x 0 := {n, x n =0} = n x n 0 3-sparse 13

19 Sparsity: definition Not sparse A vector is sparse if it has (many) zero coefficients k-sparse if it has k nonzero coefficients Symbolic representation as column vector Support = indices of nonzero components Sparsity measured with L0 pseudo-norm x 0 := {n, x n =0} = x n 0 n In french: sparse -> «creux», «parcimonieux» sparsity, sparseness -> «parcimonie», «sparsité» 3-sparse 13

20 Sparsity and subset selection Under-determined system Infinitely many solutions If vector is sparse: b ~ A x 14

21 Sparsity and subset selection Under-determined system b ~ A Infinitely many solutions If vector is sparse: If support is known (and columns independent) x 14

22 Sparsity and subset selection Under-determined system Infinitely many solutions If vector is sparse: If support is known (and columns independent) b ~ nonzero values characterized by (over)determined linear problem A x b ~ 14

23 Sparsity and subset selection Under-determined system Infinitely many solutions If vector is sparse: If support is known (and columns independent) nonzero values characterized by (over)determined linear problem If support is unknown Main issue = finding the support! This is the subset selection problem b ~ b ~ A x 14

24 Sparsity and subset selection Under-determined system Infinitely many solutions If vector is sparse: If support is known (and columns independent) nonzero values characterized by (over)determined linear problem If support is unknown Main issue = finding the support! This is the subset selection problem Objectives of the tutorial b ~ b ~ Well-posedness of subset selection Efficient subset selection algorithms = pursuit algorithms Stability guarantees of pursuits A x 14

25 Inverse Problems & Sparsity: Mathematical foundations Bottleneck : Ill-posedness when fewer equations than unknowns Novelty : Well-posedness = uniqueness of sparse solution: x 0, x 1 Ax 0 = Ax 1 x 0 = x 1 if are sufficiently sparse, then Ax 0 = Ax 1 x 0 = x 1 Recovery of with practical pursuit algorithms x 0 Thresholding, Matching Pursuits, Minimisation of Lp norms p<=1,... 15

26 Sparse models & data compression -

27 Relevance of the sparsity assumption Audio : time-frequency representations (MP3) ANALYSIS SYNTHESIS Black = zero Images : wavelet transform (JPEG2000) ANALYSIS SYNTHESIS Gray = zero 17

28 Mathematical expression of the sparsity assumption Signal / image = high dimensional vector Definition: Atoms: basis vectors ex: time-frequency atoms, wavelets Dictionary: collection of atoms y 2 R N ' k 2 R N {' k } 1applekappleK =[' k ] 1applekappleK matrix which columns are the atoms Sparse signal model = combination of few atoms y X k x k ' k = x 18

29 Sparsity & compression Full vector Sparse vector y N x Key practical issues: choose dictionary 19

30 Sparsity & compression Full vector Sparse vector N entries = N floats N y x Key practical issues: choose dictionary 19

31 Sparsity & compression Full vector Sparse vector N entries = N floats N y x k N nonzero entries = k floats Key practical issues: choose dictionary 19

32 Sparsity & compression Full vector Sparse vector N entries = N floats N y x k N nonzero entries = k floats + k positions among N = bits log 2 N k k log 2 N k Key practical issues: choose dictionary 19

33 Sparse recovery: well-posedness -

34 Example: Inpainting Problem Unknown image with N pixels y 2 R N y Sparse Model in wavelet domain wavelets coefficients are sparse x T y sparse representation of unknown image Measurement matrix y x b = My b M x A := M 21

35 Signal Processing Vocabulary x Unknown representation, sources,... Forward linear model b Ax Known linear system: A dictionary, mixing matrix, sensing system... Decomposition Reconstruction Separation Observed data: b signal, image, mixture of sources,... 22

36 (My Basic Understanding of) Machine Learning Vocabulary w Regression Unknown xcoeffs representation, sources,... Forward linear model b Ax y = Xw Known Design linear matrix system: A dictionary, mixing matrix, sensing system... X X y Observation Observed data: b signal, image, mixture of sources,... Decomposition Reconstruction Separation 23

37 (My Basic Understanding of) Statistics Vocabulary Regression Unknown xcoeffs representation, sources,... Forward linear model b Ax y = X Known Design linear matrix system: A dictionary, mixing matrix, sensing system... X X y Observation Observed data: b signal, image, mixture of sources,... Decomposition Reconstruction Separation 24

38 Well-posedness? What property should A satisfy such that, for any pair of k-sparse vectors x 0, x 1 Ax 0 = Ax 1 x 0 = x 1 25

39 Well-posedness = identifiability of k-sparse vectors Theorem: if every 2k columns of A are linearly independent, then for every k-sparse vectors Ax 0 = Ax 1 x 0 = x 1 x 0,x 1 26

40 Well-posedness = identifiability of k-sparse vectors Theorem: if every 2k columns of A are linearly independent, then for every k-sparse vectors Proof: define the vector Its support Ax 0 = Ax 1 x 0 = x 1 z = x 0 x 1 I := {i : z i =0} is of size at most 2k x 0,x 1 I = kzk 0 applekx 0 k 0 + kx 1 k 0 apple 2k P It is in the null space of A hence i2i z ia i = Az =0 The columns indexed by I are linearly independent hence z =0 26

41 Notions of spark / Kruskal rank Definition: spark(a) size of minimal set of linearly dependent columns Definition: Kruskal rank K-rank(A): maximal L such that every L columns linearly indep. Property K-rank(A) = spark(a) Well-posedness for k-sparse vectors iff 2k apple K-rank(A) 1 apple rank(a)... but the computation of K-rank for an arbitrary A is NP-complete 27

42 Examples / Exercices Definition: Kruskal rank K-rank(A): maximal L such that every L columns linearly indep. Small spark / Kruskal-rank if A contains two copies of the same column Largest spark: m x N «Vandermonde» matrix with 0 Random Gaussian matrix: N.... with probability one:. m 1... m N 1 C A K-rank(A) =?? i 6= m<n K-rank(A) =?? j, 8i 6= j A =(a ij ) a ij N (0, 1) K-rank(A) =?? 28

43 Sparse recovery: complexity -

44 Sparse recovery? Assuming well-posedness, how to compute the unique k-sparse solution to b = Ax 30

45 Ideal sparse approximation Input: m x N matrix A, with m < N, m-dimensional vector b Objective: find a sparsest approximation within given tolerance arg min x x 0, s.t. b Ax 31

46 Success of Ideal Sparse Approximation Theorem: if then 2k apple K-rank(A) Well-posedness for every pair of k-sparse vectors x 0,x 1 Ax 0 = Ax 1 x 0 = x 1 Recovery by L0 minimization for every k-sparse vector x 0 we have x 0 = arg min x kxk 0 s.t. Ax = Ax 0 32

47 Complexity of Ideal Sparse Approximation Naive: Brute force search Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

48 Complexity of Ideal Sparse Approximation Naive: Brute force search Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

49 Complexity of Ideal Sparse Approximation Naive: Brute force search Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

50 Complexity of Ideal Sparse Approximation Naive: Brute force search Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

51 Complexity of Ideal Sparse Approximation Naive: Brute force search Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

52 Complexity of Ideal Sparse Approximation Naive: Brute force search Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

53 Complexity of Ideal Sparse Approximation Naive: Brute force search Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

54 Complexity of Ideal Sparse Approximation Naive: Brute force search min x kb Axk 2 s.t. support(x) =I Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

55 Complexity of Ideal Sparse Approximation Naive: Brute force search min x kb Axk 2 s.t. support(x) =I Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

56 Complexity of Ideal Sparse Approximation Naive: Brute force search min x kb Axk 2 s.t. support(x) =I Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

57 Complexity of Ideal Sparse Approximation Naive: Brute force search min x kb Axk 2 s.t. support(x) =I Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

58 Complexity of Ideal Sparse Approximation Naive: Brute force search min x kb Axk 2 s.t. support(x) =I Many k-tuples to try! Theorem (Davies et al, Natarajan) Solving the L0 optimization problem is NP-complete Are there other more efficient alternatives? 33

59 Efficient Sparse Approximation With Time-Frequency Atoms Audio = superimposition of structures Example : glockenspiel b 2 R transients = short, small scale harmonic part = long, large scale Two-layer sparse model with Gabor atoms s,,f (t) = p 1 t w s s 2i ft e s,,f b 1 x x 2 34

60 Efficient Sparse Approximation With Time-Frequency Atoms Audio = superimposition of structures Example : glockenspiel 1x 1 b 2 R transients = short, small scale harmonic part = long, large scale Two-layer sparse model with Gabor atoms s,,f (t) = p 1 t w s s 2i ft e s,,f b 1 x x 2 34

61 Efficient Sparse Approximation With Time-Frequency Atoms Audio = superimposition of structures Example : glockenspiel 1x 1 b 2 R transients = short, small scale harmonic part = long, large scale Two-layer sparse model with Gabor atoms s,,f (t) = p 1 t w s s 2i ft e s,,f b 1 x x 2 2x 2 34

62 Towards Pursuit Algorithms Optimization principles -

63 Overall compromise Approximation quality Ax b 2 Ideal sparsity measure : norm 0 x 0 := {n, x n =0} = x n 0 Relaxed sparsity measures 0 <p<, x p := n n x n p 1/p 36

64 Norms when Lp norms / quasi-norms 1 p< x p =0 x =0 x p = x p,,x = convex Triangle inequality x + y p x p + y p, x, y Quasi-norms when Quasi-triangle inequality x + y p x + y p p Pseudo -norm for p=0 x + y 0 0 <p<1 2 1/p x p + y p, x, y x p p + y p p, x, y x 0 + y 0, x, y = nonconvex 37

65 Optimization problems Approximation min x b Ax 2 s.t. x p Sparsification min x x p s.t. b Ax 2 Regularization 1 min x 2 b Ax + x 2 p 38

66 Lp norms level sets Strictly convex when p>1 Convex p=1 Nonconvex p<1 Observation: the minimizer is sparse {x s.t.b = Ax} when p<=1 39

67 Global Optimization : from Principles to Algorithms Optimization principle NP-hard combinatorial Sparse representation Sparse approximation FOCUSS / IRLS 1 min x 2 Ax b x p p 0 > 0 Ax = b Ax b Iterative thresholding / proximal algo. Linear local minima convex : global minimum Lasso [Tibshirani 1996], Basis Pursuit (Denoising) [Chen, Donoho & Saunders, 1999] Linear/Quadratic programming (interior point, etc.) Homotopy method [Osborne 2000] / Least Angle Regression [Efron &al 2002] Iterative / proximal algorithms [Daubechies, de Frise, de Mol 2004, Combettes & Pesquet 2008, Beck & Teboulle ] 40

68 Greedy Pursuit Algorithms -

69 The orthonormal case: thresholding Observation: when A is orthonormal, the problem is equivalent to X min (a T n b x n ) 2 s.t. kxk 0 apple k x Let k min b Ax 2 2 s.t. x 0 k x n index the k largest inner products min a T n b max a T n b n2 k n/2 k an optimum solution is x n = a T n b,n2 k ; x n =0,n /2 k 42

70 Matching Pursuit (MP) Iterative algorithm (aka Projection Pursuit, CLEAN) Initialization Atom selection: r 0 = b i =1 n i = arg max n at n r i 1 Residual update r i = r i 1 (a T n i r i 1 )a ni Sparse approximation after k steps b = kx i=1 a T n r i i 1 a + r ni k 43

71 b V k = span(a ni, 1 apple i apple k) 44

72 Orthonormal MP (OMP) [Mallat & Zhang 93, Pati & al 94] Observation: after k iterations Approximant belongs to V k = span(a ni, 1 apple i apple k) Best approximation from = V k P Vk b = A k A + k b k = {n i, 1 i k} r k = b kx i=1 orthoprojection k a ni OMP residual update rule r k = b P Vk b 45

73 OMP Same as MP, except residual update rule Atom selection: n i = arg max n at n r i 1 Index update Residual update i = i 1 {n i } V i = span(a n,n2 i ) r i = b P Vi b 46

74 Stagewise greedy algorithms Principle select multiple atoms at a time to accelerate the process possibly prune out some atoms at each stage Example of such algorithms Morphological Component Analysis [MCA, Bobin et al] Stagewise OMP [Donoho & al] CoSAMP [Needell & Tropp] ROMP [Needell & Vershynin] Iterative Hard Thresholding [Blumensath & Davies 2008] 47

75 Overview of greedy algorithms b = Ax i + r i A =[A 1,...A N ] Matching Pursuit OMP Stagewise OMP Selection i := arg max n AT n r i 1 i := {n A T n r i 1 > i } Update i = i 1 i x i = x i 1 + A + i r i 1 r i = r i 1 A i A + i r i 1 i = i 1 i x i = A + b i r i = b A i x i MP & OMP: Mallat & Zhang 1993 StOMP: Donoho & al 2006 (similar to MCA, Bobin & al 2006) 48

76 Summary Global optimization Iterative greedy algorithms Principle 1 min x 2 Ax b x p p iterative decomposition select new components update residual r i = b Ax i Tuning quality/sparsity regularization parameter stopping criterion (nb of iterations, error level,...) x i 0 k r i Variants choice of sparsity measure p optimization algorithm initialization selection criterion (weak, stagewise...) update strategy (orthogonal...) 49

77 Sparse recovery: Provably good algorithms? -

78 Inverse problems Signal space ~ R N Set of signals of interest Nonlinear Approximation = Sparse recovery Linear projection Courtesy: M. Davies, U. Edinburgh Observation space ~ R M M<<N 51

79 Recovery analysis for inverse problem b = Ax Recoverable set for a given inversion algorithm {x = Algo1(Ax)} 52

80 Recovery analysis for inverse problem b = Ax Recoverable set for a given inversion algorithm Level sets of L0-norm 1-sparse x 0 1 {x = Algo1(Ax)} 52

81 Recovery analysis for inverse problem b = Ax Recoverable set for a given inversion algorithm Level sets of L0-norm 1-sparse 2-sparse x 0 x 0 1 k {x = Algo1(Ax)} 52

82 Recovery analysis for inverse problem b = Ax Recoverable set for a given inversion algorithm Level sets of L0-norm 1-sparse 2-sparse 3-sparse... x 0 x 0 1 k {x = Algo1(Ax)} 52

83 Recovery analysis for inverse problem b = Ax Recoverable set for a given inversion algorithm Level sets of L0-norm 1-sparse 2-sparse 3-sparse... x 0 x 0 1 k 53

84 Recovery analysis for inverse problem b = Ax Recoverable set for a given inversion algorithm Level sets of L0-norm 1-sparse 2-sparse 3-sparse... {x = Algo2(Ax)} x 0 x 0 1 k 53

85 Recovery analysis for inverse problem b = Ax Recoverable set for a given inversion algorithm Level sets of L0-norm 1-sparse 2-sparse 3-sparse... {x = Algo2(Ax)} x 0 x 0 1 k 53

86 Equivalence between L0, L1, OMP Theorem : assume that b = Ax 0 if x 0 0 k 0 (A) then x 0 = x 0 if x 0 0 k 1 (A) then x 0 = x 1 where x p = arg min Ax=Ax 0 x p Donoho & Huo 01 : pair of bases, coherence Donoho & Elad, Gribonval & Nielsen 2003 : dictionary, coherence Tropp 2004 : Orthonormal Matching Pursuit, cumulative coherence Candes, Romberg, Tao 2004 : random dictionaries, restricted isometry constants 54

87 Robust / stable sparse recovery? -

88 Stable identifiability Identifiability of 1-sparse vectors, with A 2x3 matrix Robust Not robust Not identifiable Beyond well-posedness: need good conditioning 56

89 Quantitative well-posedness: Coherence Coherence of A =[a 1...a N ] µ(a) := max i6=j a i a i 2, a j a j 2 µ 1 µ. 1 µ =1 Incoherent Coherent Perfectly coherent Small coherence implies large Kruskal rank 57

Inverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France.

Inverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France. Inverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Overview of the course Introduction sparsity & data compression inverse problems

More information

An Introduction to Sparse Approximation

An Introduction to Sparse Approximation An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,

More information

Introduction to Compressed Sensing

Introduction to Compressed Sensing Introduction to Compressed Sensing Alejandro Parada, Gonzalo Arce University of Delaware August 25, 2016 Motivation: Classical Sampling 1 Motivation: Classical Sampling Issues Some applications Radar Spectral

More information

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images!

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images! Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images! Alfredo Nava-Tudela John J. Benedetto, advisor 1 Happy birthday Lucía! 2 Outline - Problem: Find sparse solutions

More information

Sparsity in Underdetermined Systems

Sparsity in Underdetermined Systems Sparsity in Underdetermined Systems Department of Statistics Stanford University August 19, 2005 Classical Linear Regression Problem X n y p n 1 > Given predictors and response, y Xβ ε = + ε N( 0, σ 2

More information

Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees

Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html Acknowledgement: this slides is based on Prof. Emmanuel Candes and Prof. Wotao Yin

More information

GREEDY SIGNAL RECOVERY REVIEW

GREEDY SIGNAL RECOVERY REVIEW GREEDY SIGNAL RECOVERY REVIEW DEANNA NEEDELL, JOEL A. TROPP, ROMAN VERSHYNIN Abstract. The two major approaches to sparse recovery are L 1-minimization and greedy methods. Recently, Needell and Vershynin

More information

Generalized Orthogonal Matching Pursuit- A Review and Some

Generalized Orthogonal Matching Pursuit- A Review and Some Generalized Orthogonal Matching Pursuit- A Review and Some New Results Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur, INDIA Table of Contents

More information

Sparse & Redundant Signal Representation, and its Role in Image Processing

Sparse & Redundant Signal Representation, and its Role in Image Processing Sparse & Redundant Signal Representation, and its Role in Michael Elad The CS Department The Technion Israel Institute of technology Haifa 3000, Israel Wave 006 Wavelet and Applications Ecole Polytechnique

More information

c 2011 International Press Vol. 18, No. 1, pp , March DENNIS TREDE

c 2011 International Press Vol. 18, No. 1, pp , March DENNIS TREDE METHODS AND APPLICATIONS OF ANALYSIS. c 2011 International Press Vol. 18, No. 1, pp. 105 110, March 2011 007 EXACT SUPPORT RECOVERY FOR LINEAR INVERSE PROBLEMS WITH SPARSITY CONSTRAINTS DENNIS TREDE Abstract.

More information

Inverse problems, Dictionary based Signal Models and Compressed Sensing

Inverse problems, Dictionary based Signal Models and Compressed Sensing Inverse problems, Dictionary based Signal Models and Compressed Sensing Rémi Gribonval METISS project-team (audio signal processing, speech recognition, source separation) INRIA, Rennes, France Ecole d

More information

Greedy Signal Recovery and Uniform Uncertainty Principles

Greedy Signal Recovery and Uniform Uncertainty Principles Greedy Signal Recovery and Uniform Uncertainty Principles SPIE - IE 2008 Deanna Needell Joint work with Roman Vershynin UC Davis, January 2008 Greedy Signal Recovery and Uniform Uncertainty Principles

More information

Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images

Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images Alfredo Nava-Tudela ant@umd.edu John J. Benedetto Department of Mathematics jjb@umd.edu Abstract In this project we are

More information

Pre-weighted Matching Pursuit Algorithms for Sparse Recovery

Pre-weighted Matching Pursuit Algorithms for Sparse Recovery Journal of Information & Computational Science 11:9 (214) 2933 2939 June 1, 214 Available at http://www.joics.com Pre-weighted Matching Pursuit Algorithms for Sparse Recovery Jingfei He, Guiling Sun, Jie

More information

MLCC 2018 Variable Selection and Sparsity. Lorenzo Rosasco UNIGE-MIT-IIT

MLCC 2018 Variable Selection and Sparsity. Lorenzo Rosasco UNIGE-MIT-IIT MLCC 2018 Variable Selection and Sparsity Lorenzo Rosasco UNIGE-MIT-IIT Outline Variable Selection Subset Selection Greedy Methods: (Orthogonal) Matching Pursuit Convex Relaxation: LASSO & Elastic Net

More information

A simple test to check the optimality of sparse signal approximations

A simple test to check the optimality of sparse signal approximations A simple test to check the optimality of sparse signal approximations Rémi Gribonval, Rosa Maria Figueras I Ventura, Pierre Vergheynst To cite this version: Rémi Gribonval, Rosa Maria Figueras I Ventura,

More information

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation Alfredo Nava-Tudela John J. Benedetto, advisor 5/10/11 AMSC 663/664 1 Problem Let A be an n

More information

Uniqueness Conditions for A Class of l 0 -Minimization Problems

Uniqueness Conditions for A Class of l 0 -Minimization Problems Uniqueness Conditions for A Class of l 0 -Minimization Problems Chunlei Xu and Yun-Bin Zhao October, 03, Revised January 04 Abstract. We consider a class of l 0 -minimization problems, which is to search

More information

The Analysis Cosparse Model for Signals and Images

The Analysis Cosparse Model for Signals and Images The Analysis Cosparse Model for Signals and Images Raja Giryes Computer Science Department, Technion. The research leading to these results has received funding from the European Research Council under

More information

New Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit

New Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence

More information

MATCHING PURSUIT WITH STOCHASTIC SELECTION

MATCHING PURSUIT WITH STOCHASTIC SELECTION 2th European Signal Processing Conference (EUSIPCO 22) Bucharest, Romania, August 27-3, 22 MATCHING PURSUIT WITH STOCHASTIC SELECTION Thomas Peel, Valentin Emiya, Liva Ralaivola Aix-Marseille Université

More information

Sketching for Large-Scale Learning of Mixture Models

Sketching for Large-Scale Learning of Mixture Models Sketching for Large-Scale Learning of Mixture Models Nicolas Keriven Université Rennes 1, Inria Rennes Bretagne-atlantique Adv. Rémi Gribonval Outline Introduction Practical Approach Results Theoretical

More information

Stability and Robustness of Weak Orthogonal Matching Pursuits

Stability and Robustness of Weak Orthogonal Matching Pursuits Stability and Robustness of Weak Orthogonal Matching Pursuits Simon Foucart, Drexel University Abstract A recent result establishing, under restricted isometry conditions, the success of sparse recovery

More information

SPARSE signal representations have gained popularity in recent

SPARSE signal representations have gained popularity in recent 6958 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 10, OCTOBER 2011 Blind Compressed Sensing Sivan Gleichman and Yonina C. Eldar, Senior Member, IEEE Abstract The fundamental principle underlying

More information

Compressed sensing. Or: the equation Ax = b, revisited. Terence Tao. Mahler Lecture Series. University of California, Los Angeles

Compressed sensing. Or: the equation Ax = b, revisited. Terence Tao. Mahler Lecture Series. University of California, Los Angeles Or: the equation Ax = b, revisited University of California, Los Angeles Mahler Lecture Series Acquiring signals Many types of real-world signals (e.g. sound, images, video) can be viewed as an n-dimensional

More information

Thresholds for the Recovery of Sparse Solutions via L1 Minimization

Thresholds for the Recovery of Sparse Solutions via L1 Minimization Thresholds for the Recovery of Sparse Solutions via L Minimization David L. Donoho Department of Statistics Stanford University 39 Serra Mall, Sequoia Hall Stanford, CA 9435-465 Email: donoho@stanford.edu

More information

Sparse Optimization Lecture: Sparse Recovery Guarantees

Sparse Optimization Lecture: Sparse Recovery Guarantees Those who complete this lecture will know Sparse Optimization Lecture: Sparse Recovery Guarantees Sparse Optimization Lecture: Sparse Recovery Guarantees Instructor: Wotao Yin Department of Mathematics,

More information

A tutorial on sparse modeling. Outline:

A tutorial on sparse modeling. Outline: A tutorial on sparse modeling. Outline: 1. Why? 2. What? 3. How. 4. no really, why? Sparse modeling is a component in many state of the art signal processing and machine learning tasks. image processing

More information

Multipath Matching Pursuit

Multipath Matching Pursuit Multipath Matching Pursuit Submitted to IEEE trans. on Information theory Authors: S. Kwon, J. Wang, and B. Shim Presenter: Hwanchol Jang Multipath is investigated rather than a single path for a greedy

More information

Introduction to Sparsity. Xudong Cao, Jake Dreamtree & Jerry 04/05/2012

Introduction to Sparsity. Xudong Cao, Jake Dreamtree & Jerry 04/05/2012 Introduction to Sparsity Xudong Cao, Jake Dreamtree & Jerry 04/05/2012 Outline Understanding Sparsity Total variation Compressed sensing(definition) Exact recovery with sparse prior(l 0 ) l 1 relaxation

More information

Sensing systems limited by constraints: physical size, time, cost, energy

Sensing systems limited by constraints: physical size, time, cost, energy Rebecca Willett Sensing systems limited by constraints: physical size, time, cost, energy Reduce the number of measurements needed for reconstruction Higher accuracy data subject to constraints Original

More information

Compressive Sensing and Beyond

Compressive Sensing and Beyond Compressive Sensing and Beyond Sohail Bahmani Gerorgia Tech. Signal Processing Compressed Sensing Signal Models Classics: bandlimited The Sampling Theorem Any signal with bandwidth B can be recovered

More information

Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise

Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise Orthogonal Matching Pursuit for Sparse Signal Recovery With Noise The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published

More information

of Orthogonal Matching Pursuit

of Orthogonal Matching Pursuit A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit Qun Mo arxiv:50.0708v [cs.it] 8 Jan 205 Abstract We shall show that if the restricted isometry constant (RIC) δ s+ (A) of the measurement

More information

Sparse analysis Lecture III: Dictionary geometry and greedy algorithms

Sparse analysis Lecture III: Dictionary geometry and greedy algorithms Sparse analysis Lecture III: Dictionary geometry and greedy algorithms Anna C. Gilbert Department of Mathematics University of Michigan Intuition from ONB Key step in algorithm: r, ϕ j = x c i ϕ i, ϕ j

More information

Signal Recovery From Incomplete and Inaccurate Measurements via Regularized Orthogonal Matching Pursuit

Signal Recovery From Incomplete and Inaccurate Measurements via Regularized Orthogonal Matching Pursuit Signal Recovery From Incomplete and Inaccurate Measurements via Regularized Orthogonal Matching Pursuit Deanna Needell and Roman Vershynin Abstract We demonstrate a simple greedy algorithm that can reliably

More information

Signal Sparsity Models: Theory and Applications

Signal Sparsity Models: Theory and Applications Signal Sparsity Models: Theory and Applications Raja Giryes Computer Science Department, Technion Michael Elad, Technion, Haifa Israel Sangnam Nam, CMI, Marseille France Remi Gribonval, INRIA, Rennes France

More information

Machine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013

Machine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 Machine Learning for Signal Processing Sparse and Overcomplete Representations Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 1 Key Topics in this Lecture Basics Component-based representations

More information

CoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles

CoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles CoSaMP: Greedy Signal Recovery and Uniform Uncertainty Principles SIAM Student Research Conference Deanna Needell Joint work with Roman Vershynin and Joel Tropp UC Davis, May 2008 CoSaMP: Greedy Signal

More information

Compressed Sensing. 1 Introduction. 2 Design of Measurement Matrices

Compressed Sensing. 1 Introduction. 2 Design of Measurement Matrices Compressed Sensing Yonina C. Eldar Electrical Engineering Department, Technion-Israel Institute of Technology, Haifa, Israel, 32000 1 Introduction Compressed sensing (CS) is an exciting, rapidly growing

More information

Gradient Descent with Sparsification: An iterative algorithm for sparse recovery with restricted isometry property

Gradient Descent with Sparsification: An iterative algorithm for sparse recovery with restricted isometry property : An iterative algorithm for sparse recovery with restricted isometry property Rahul Garg grahul@us.ibm.com Rohit Khandekar rohitk@us.ibm.com IBM T. J. Watson Research Center, 0 Kitchawan Road, Route 34,

More information

The Iteration-Tuned Dictionary for Sparse Representations

The Iteration-Tuned Dictionary for Sparse Representations The Iteration-Tuned Dictionary for Sparse Representations Joaquin Zepeda #1, Christine Guillemot #2, Ewa Kijak 3 # INRIA Centre Rennes - Bretagne Atlantique Campus de Beaulieu, 35042 Rennes Cedex, FRANCE

More information

The Sparsest Solution of Underdetermined Linear System by l q minimization for 0 < q 1

The Sparsest Solution of Underdetermined Linear System by l q minimization for 0 < q 1 The Sparsest Solution of Underdetermined Linear System by l q minimization for 0 < q 1 Simon Foucart Department of Mathematics Vanderbilt University Nashville, TN 3784. Ming-Jun Lai Department of Mathematics,

More information

Recovery of Sparse Signals Using Multiple Orthogonal Least Squares

Recovery of Sparse Signals Using Multiple Orthogonal Least Squares Recovery of Sparse Signals Using Multiple Orthogonal east Squares Jian Wang, Ping i Department of Statistics and Biostatistics arxiv:40.505v [stat.me] 9 Oct 04 Department of Computer Science Rutgers University

More information

Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach

Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach Boaz Nadler The Weizmann Institute of Science Israel Joint works with Inbal Horev, Ronen Basri, Meirav Galun and Ery Arias-Castro

More information

Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation

Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 19, NO. 12, DECEMBER 2008 2009 Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation Yuanqing Li, Member, IEEE, Andrzej Cichocki,

More information

A new method on deterministic construction of the measurement matrix in compressed sensing

A new method on deterministic construction of the measurement matrix in compressed sensing A new method on deterministic construction of the measurement matrix in compressed sensing Qun Mo 1 arxiv:1503.01250v1 [cs.it] 4 Mar 2015 Abstract Construction on the measurement matrix A is a central

More information

Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego

Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego Bhaskar Rao Department of Electrical and Computer Engineering University of California, San Diego 1 Outline Course Outline Motivation for Course Sparse Signal Recovery Problem Applications Computational

More information

COMPARATIVE ANALYSIS OF ORTHOGONAL MATCHING PURSUIT AND LEAST ANGLE REGRESSION

COMPARATIVE ANALYSIS OF ORTHOGONAL MATCHING PURSUIT AND LEAST ANGLE REGRESSION COMPARATIVE ANALYSIS OF ORTHOGONAL MATCHING PURSUIT AND LEAST ANGLE REGRESSION By Mazin Abdulrasool Hameed A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for

More information

Convex Optimization and l 1 -minimization

Convex Optimization and l 1 -minimization Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l

More information

Machine Learning for Signal Processing Sparse and Overcomplete Representations

Machine Learning for Signal Processing Sparse and Overcomplete Representations Machine Learning for Signal Processing Sparse and Overcomplete Representations Abelino Jimenez (slides from Bhiksha Raj and Sourish Chaudhuri) Oct 1, 217 1 So far Weights Data Basis Data Independent ICA

More information

Abstract This paper is about the efficient solution of large-scale compressed sensing problems.

Abstract This paper is about the efficient solution of large-scale compressed sensing problems. Noname manuscript No. (will be inserted by the editor) Optimization for Compressed Sensing: New Insights and Alternatives Robert Vanderbei and Han Liu and Lie Wang Received: date / Accepted: date Abstract

More information

Exact Low-rank Matrix Recovery via Nonconvex M p -Minimization

Exact Low-rank Matrix Recovery via Nonconvex M p -Minimization Exact Low-rank Matrix Recovery via Nonconvex M p -Minimization Lingchen Kong and Naihua Xiu Department of Applied Mathematics, Beijing Jiaotong University, Beijing, 100044, People s Republic of China E-mail:

More information

Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery

Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan Connection between... Sparse Approximation and Compressed

More information

Large-Scale L1-Related Minimization in Compressive Sensing and Beyond

Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Arizona State University March

More information

Compressed Sensing: Extending CLEAN and NNLS

Compressed Sensing: Extending CLEAN and NNLS Compressed Sensing: Extending CLEAN and NNLS Ludwig Schwardt SKA South Africa (KAT Project) Calibration & Imaging Workshop Socorro, NM, USA 31 March 2009 Outline 1 Compressed Sensing (CS) Introduction

More information

Signal Recovery, Uncertainty Relations, and Minkowski Dimension

Signal Recovery, Uncertainty Relations, and Minkowski Dimension Signal Recovery, Uncertainty Relations, and Minkowski Dimension Helmut Bőlcskei ETH Zurich December 2013 Joint work with C. Aubel, P. Kuppinger, G. Pope, E. Riegler, D. Stotz, and C. Studer Aim of this

More information

Complementary Matching Pursuit Algorithms for Sparse Approximation

Complementary Matching Pursuit Algorithms for Sparse Approximation Complementary Matching Pursuit Algorithms for Sparse Approximation Gagan Rath and Christine Guillemot IRISA-INRIA, Campus de Beaulieu 35042 Rennes, France phone: +33.2.99.84.75.26 fax: +33.2.99.84.71.71

More information

Applied Machine Learning for Biomedical Engineering. Enrico Grisan

Applied Machine Learning for Biomedical Engineering. Enrico Grisan Applied Machine Learning for Biomedical Engineering Enrico Grisan enrico.grisan@dei.unipd.it Data representation To find a representation that approximates elements of a signal class with a linear combination

More information

Uncertainty principles and sparse approximation

Uncertainty principles and sparse approximation Uncertainty principles and sparse approximation In this lecture, we will consider the special case where the dictionary Ψ is composed of a pair of orthobases. We will see that our ability to find a sparse

More information

Homotopy methods based on l 0 norm for the compressed sensing problem

Homotopy methods based on l 0 norm for the compressed sensing problem Homotopy methods based on l 0 norm for the compressed sensing problem Wenxing Zhu, Zhengshan Dong Center for Discrete Mathematics and Theoretical Computer Science, Fuzhou University, Fuzhou 350108, China

More information

A Generalized Uncertainty Principle and Sparse Representation in Pairs of Bases

A Generalized Uncertainty Principle and Sparse Representation in Pairs of Bases 2558 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 9, SEPTEMBER 2002 A Generalized Uncertainty Principle Sparse Representation in Pairs of Bases Michael Elad Alfred M Bruckstein Abstract An elementary

More information

ORTHOGONAL matching pursuit (OMP) is the canonical

ORTHOGONAL matching pursuit (OMP) is the canonical IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 56, NO. 9, SEPTEMBER 2010 4395 Analysis of Orthogonal Matching Pursuit Using the Restricted Isometry Property Mark A. Davenport, Member, IEEE, and Michael

More information

A Tutorial on Compressive Sensing. Simon Foucart Drexel University / University of Georgia

A Tutorial on Compressive Sensing. Simon Foucart Drexel University / University of Georgia A Tutorial on Compressive Sensing Simon Foucart Drexel University / University of Georgia CIMPA13 New Trends in Applied Harmonic Analysis Mar del Plata, Argentina, 5-16 August 2013 This minicourse acts

More information

5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER /$ IEEE

5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER /$ IEEE 5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER 2009 Uncertainty Relations for Shift-Invariant Analog Signals Yonina C. Eldar, Senior Member, IEEE Abstract The past several years

More information

On the Minimization Over Sparse Symmetric Sets: Projections, O. Projections, Optimality Conditions and Algorithms

On the Minimization Over Sparse Symmetric Sets: Projections, O. Projections, Optimality Conditions and Algorithms On the Minimization Over Sparse Symmetric Sets: Projections, Optimality Conditions and Algorithms Amir Beck Technion - Israel Institute of Technology Haifa, Israel Based on joint work with Nadav Hallak

More information

Sparsest Solutions of Underdetermined Linear Systems via l q -minimization for 0 < q 1

Sparsest Solutions of Underdetermined Linear Systems via l q -minimization for 0 < q 1 Sparsest Solutions of Underdetermined Linear Systems via l q -minimization for 0 < q 1 Simon Foucart Department of Mathematics Vanderbilt University Nashville, TN 3740 Ming-Jun Lai Department of Mathematics

More information

Basis Pursuit Denoising and the Dantzig Selector

Basis Pursuit Denoising and the Dantzig Selector BPDN and DS p. 1/16 Basis Pursuit Denoising and the Dantzig Selector West Coast Optimization Meeting University of Washington Seattle, WA, April 28 29, 2007 Michael Friedlander and Michael Saunders Dept

More information

Uniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit

Uniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit Uniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit arxiv:0707.4203v2 [math.na] 14 Aug 2007 Deanna Needell Department of Mathematics University of California,

More information

Sparse Solutions of an Undetermined Linear System

Sparse Solutions of an Undetermined Linear System 1 Sparse Solutions of an Undetermined Linear System Maddullah Almerdasy New York University Tandon School of Engineering arxiv:1702.07096v1 [math.oc] 23 Feb 2017 Abstract This work proposes a research

More information

Algorithms for sparse analysis Lecture I: Background on sparse approximation

Algorithms for sparse analysis Lecture I: Background on sparse approximation Algorithms for sparse analysis Lecture I: Background on sparse approximation Anna C. Gilbert Department of Mathematics University of Michigan Tutorial on sparse approximations and algorithms Compress data

More information

Bayesian Methods for Sparse Signal Recovery

Bayesian Methods for Sparse Signal Recovery Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Jason Palmer, Zhilin Zhang and Ritwik Giri Motivation Motivation Sparse Signal Recovery

More information

Sparse linear models

Sparse linear models Sparse linear models Optimization-Based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_spring16 Carlos Fernandez-Granda 2/22/2016 Introduction Linear transforms Frequency representation Short-time

More information

MATCHING-PURSUIT DICTIONARY PRUNING FOR MPEG-4 VIDEO OBJECT CODING

MATCHING-PURSUIT DICTIONARY PRUNING FOR MPEG-4 VIDEO OBJECT CODING MATCHING-PURSUIT DICTIONARY PRUNING FOR MPEG-4 VIDEO OBJECT CODING Yannick Morvan, Dirk Farin University of Technology Eindhoven 5600 MB Eindhoven, The Netherlands email: {y.morvan;d.s.farin}@tue.nl Peter

More information

Model-Based Compressive Sensing for Signal Ensembles. Marco F. Duarte Volkan Cevher Richard G. Baraniuk

Model-Based Compressive Sensing for Signal Ensembles. Marco F. Duarte Volkan Cevher Richard G. Baraniuk Model-Based Compressive Sensing for Signal Ensembles Marco F. Duarte Volkan Cevher Richard G. Baraniuk Concise Signal Structure Sparse signal: only K out of N coordinates nonzero model: union of K-dimensional

More information

Optimization methods

Optimization methods Lecture notes 3 February 8, 016 1 Introduction Optimization methods In these notes we provide an overview of a selection of optimization methods. We focus on methods which rely on first-order information,

More information

Tutorial: Sparse Signal Processing Part 1: Sparse Signal Representation. Pier Luigi Dragotti Imperial College London

Tutorial: Sparse Signal Processing Part 1: Sparse Signal Representation. Pier Luigi Dragotti Imperial College London Tutorial: Sparse Signal Processing Part 1: Sparse Signal Representation Pier Luigi Dragotti Imperial College London Outline Part 1: Sparse Signal Representation ~90min Part 2: Sparse Sampling ~90min 2

More information

A Review of Sparsity-Based Methods for Analysing Radar Returns from Helicopter Rotor Blades

A Review of Sparsity-Based Methods for Analysing Radar Returns from Helicopter Rotor Blades A Review of Sparsity-Based Methods for Analysing Radar Returns from Helicopter Rotor Blades Ngoc Hung Nguyen 1, Hai-Tan Tran 2, Kutluyıl Doğançay 1 and Rocco Melino 2 1 University of South Australia 2

More information

SIGNAL SEPARATION USING RE-WEIGHTED AND ADAPTIVE MORPHOLOGICAL COMPONENT ANALYSIS

SIGNAL SEPARATION USING RE-WEIGHTED AND ADAPTIVE MORPHOLOGICAL COMPONENT ANALYSIS TR-IIS-4-002 SIGNAL SEPARATION USING RE-WEIGHTED AND ADAPTIVE MORPHOLOGICAL COMPONENT ANALYSIS GUAN-JU PENG AND WEN-LIANG HWANG Feb. 24, 204 Technical Report No. TR-IIS-4-002 http://www.iis.sinica.edu.tw/page/library/techreport/tr204/tr4.html

More information

Data Sparse Matrix Computation - Lecture 20

Data Sparse Matrix Computation - Lecture 20 Data Sparse Matrix Computation - Lecture 20 Yao Cheng, Dongping Qi, Tianyi Shi November 9, 207 Contents Introduction 2 Theorems on Sparsity 2. Example: A = [Φ Ψ]......................... 2.2 General Matrix

More information

CoSaMP. Iterative signal recovery from incomplete and inaccurate samples. Joel A. Tropp

CoSaMP. Iterative signal recovery from incomplete and inaccurate samples. Joel A. Tropp CoSaMP Iterative signal recovery from incomplete and inaccurate samples Joel A. Tropp Applied & Computational Mathematics California Institute of Technology jtropp@acm.caltech.edu Joint with D. Needell

More information

IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER

IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER 2015 1239 Preconditioning for Underdetermined Linear Systems with Sparse Solutions Evaggelia Tsiligianni, StudentMember,IEEE, Lisimachos P. Kondi,

More information

Morphological Diversity and Source Separation

Morphological Diversity and Source Separation Morphological Diversity and Source Separation J. Bobin, Y. Moudden, J.-L. Starck, and M. Elad Abstract This paper describes a new method for blind source separation, adapted to the case of sources having

More information

Compressed Sensing and Neural Networks

Compressed Sensing and Neural Networks and Jan Vybíral (Charles University & Czech Technical University Prague, Czech Republic) NOMAD Summer Berlin, September 25-29, 2017 1 / 31 Outline Lasso & Introduction Notation Training the network Applications

More information

EUSIPCO

EUSIPCO EUSIPCO 013 1569746769 SUBSET PURSUIT FOR ANALYSIS DICTIONARY LEARNING Ye Zhang 1,, Haolong Wang 1, Tenglong Yu 1, Wenwu Wang 1 Department of Electronic and Information Engineering, Nanchang University,

More information

Simultaneous Sparsity

Simultaneous Sparsity Simultaneous Sparsity Joel A. Tropp Anna C. Gilbert Martin J. Strauss {jtropp annacg martinjs}@umich.edu Department of Mathematics The University of Michigan 1 Simple Sparse Approximation Work in the d-dimensional,

More information

Copyright by Joel Aaron Tropp 2004

Copyright by Joel Aaron Tropp 2004 Copyright by Joel Aaron Tropp 2004 The Dissertation Committee for Joel Aaron Tropp certifies that this is the approved version of the following dissertation: Topics in Sparse Approximation Committee: Inderjit

More information

Constructing Explicit RIP Matrices and the Square-Root Bottleneck

Constructing Explicit RIP Matrices and the Square-Root Bottleneck Constructing Explicit RIP Matrices and the Square-Root Bottleneck Ryan Cinoman July 18, 2018 Ryan Cinoman Constructing Explicit RIP Matrices July 18, 2018 1 / 36 Outline 1 Introduction 2 Restricted Isometry

More information

Robust Sparse Recovery via Non-Convex Optimization

Robust Sparse Recovery via Non-Convex Optimization Robust Sparse Recovery via Non-Convex Optimization Laming Chen and Yuantao Gu Department of Electronic Engineering, Tsinghua University Homepage: http://gu.ee.tsinghua.edu.cn/ Email: gyt@tsinghua.edu.cn

More information

Restricted Strong Convexity Implies Weak Submodularity

Restricted Strong Convexity Implies Weak Submodularity Restricted Strong Convexity Implies Weak Submodularity Ethan R. Elenberg Rajiv Khanna Alexandros G. Dimakis Department of Electrical and Computer Engineering The University of Texas at Austin {elenberg,rajivak}@utexas.edu

More information

Randomness-in-Structured Ensembles for Compressed Sensing of Images

Randomness-in-Structured Ensembles for Compressed Sensing of Images Randomness-in-Structured Ensembles for Compressed Sensing of Images Abdolreza Abdolhosseini Moghadam Dep. of Electrical and Computer Engineering Michigan State University Email: abdolhos@msu.edu Hayder

More information

Robust multichannel sparse recovery

Robust multichannel sparse recovery Robust multichannel sparse recovery Esa Ollila Department of Signal Processing and Acoustics Aalto University, Finland SUPELEC, Feb 4th, 2015 1 Introduction 2 Nonparametric sparse recovery 3 Simulation

More information

Reconstruction from Anisotropic Random Measurements

Reconstruction from Anisotropic Random Measurements Reconstruction from Anisotropic Random Measurements Mark Rudelson and Shuheng Zhou The University of Michigan, Ann Arbor Coding, Complexity, and Sparsity Workshop, 013 Ann Arbor, Michigan August 7, 013

More information

Motivation Sparse Signal Recovery is an interesting area with many potential applications. Methods developed for solving sparse signal recovery proble

Motivation Sparse Signal Recovery is an interesting area with many potential applications. Methods developed for solving sparse signal recovery proble Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Zhilin Zhang and Ritwik Giri Motivation Sparse Signal Recovery is an interesting

More information

Uniqueness and Uncertainty

Uniqueness and Uncertainty Chapter 2 Uniqueness and Uncertainty We return to the basic problem (P 0 ), which is at the core of our discussion, (P 0 ) : min x x 0 subject to b = Ax. While we shall refer hereafter to this problem

More information

Recent Developments in Compressed Sensing

Recent Developments in Compressed Sensing Recent Developments in Compressed Sensing M. Vidyasagar Distinguished Professor, IIT Hyderabad m.vidyasagar@iith.ac.in, www.iith.ac.in/ m vidyasagar/ ISL Seminar, Stanford University, 19 April 2018 Outline

More information

Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization

Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization Noname manuscript No. (will be inserted by the editor) Necessary and Sufficient Conditions of Solution Uniqueness in 1-Norm Minimization Hui Zhang Wotao Yin Lizhi Cheng Received: / Accepted: Abstract This

More information

AN INTRODUCTION TO COMPRESSIVE SENSING

AN INTRODUCTION TO COMPRESSIVE SENSING AN INTRODUCTION TO COMPRESSIVE SENSING Rodrigo B. Platte School of Mathematical and Statistical Sciences APM/EEE598 Reverse Engineering of Complex Dynamical Networks OUTLINE 1 INTRODUCTION 2 INCOHERENCE

More information

Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit

Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit Claremont Colleges Scholarship @ Claremont CMC Faculty Publications and Research CMC Faculty Scholarship 6-5-2008 Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit

More information

Block-sparse Solutions using Kernel Block RIP and its Application to Group Lasso

Block-sparse Solutions using Kernel Block RIP and its Application to Group Lasso Block-sparse Solutions using Kernel Block RIP and its Application to Group Lasso Rahul Garg IBM T.J. Watson research center grahul@us.ibm.com Rohit Khandekar IBM T.J. Watson research center rohitk@us.ibm.com

More information