Sparse & Redundant Signal Representation, and its Role in Image Processing

Size: px
Start display at page:

Download "Sparse & Redundant Signal Representation, and its Role in Image Processing"

Transcription

1 Sparse & Redundant Signal Representation, and its Role in Michael Elad The CS Department The Technion Israel Institute of technology Haifa 3000, Israel Wave 006 Wavelet and Applications Ecole Polytechnique Federale de Lausanne (EPFL) July 10 th 14 th, 006 D x

2 Today s Talk is About Sparsity and Redundancy We will try to show today that Sparsity & Redundancy can be used to design new/renewed & powerful signal/image processing tools (e.g., transforms, priors, models, compression, ), The obtained machinery works very well we will show these ideas deployed to image processing applications. D x

3 Agenda 1. A Visit to Sparseland Motivating Sparsity & Overcompleteness. Problem 1: Transforms & Regularizations How & why should this work? 3. Problem : What About D? The quest for the origin of signals Welcome to Sparseland 4. Problem 3: Applications Image filling in, denoising, separation, compression, D x 3

4 Generating Signals in Sparseland N K A fixed Dictionary D M A sparse & random vector x N Every column in D (dictionary) is a prototype signal (Atom). The vector is generated randomly with few non-zeros in random locations and random values. D x 4

5 Sparseland Signals Are Special Multiply by D x M D Simple: Every generated signal is built as a linear combination of few atoms from our dictionary D Rich: A general model: the obtained signals are a special type mixtureof-gaussians (or Laplacians). D x 5

6 Transforms in Sparseland? Assume that x is known to emerge from M.. We desire simplicity, independence, and expressiveness. How about Given x, find the that generated it in M? D M x R N T K R D D x 6

7 So, In Order to Transform We need to solve an under-determined linear system of equations: D x Among all (infinitely many) possible solutions we want the sparsest!! We will measure sparsity using the L 0 norm: Known 0 D x 7

8 Measure of Sparsity? x p p k j 1 x p p ( x) x f 1 1 As p 0 we get a count of the non-zeros in the vector 1 p p p 0 p p p < x D x 8

9 Signal s Transform in Sparseland A sparse & random vector Multiply by D x D Min s.t. 0 x D ˆ 4 Major Questions Is ˆ? Under which conditions? Are there practical ways to get ˆ? How effective are those ways? How would we get D? D x 9

10 Inverse Problems in Sparseland? Assume that x is known to emerge from M. Suppose we observe y Hx + v, a blurred and noisy version of x with v ε. How will we recover x? How about find the that generated the x again? y R M M Hx Q D x R N ˆ R K Noise D x 10

11 Inverse Problems in Sparseland? A sparse & random vector x Multiply by D D blur by H v y Hx + v Min 0 y HD s.t. ε ˆ 4 Major Questions (again!) Is ˆ? How far can it go? Are there practical ways to get ˆ? How effective are those ways? How would we get D? D x 11

12 Back Home Any Lessons? Several recent trends worth looking at: JPEG to JPEG000 - From (L -norm) KLT to wavelet and nonlinear approximation Sparsity. From Wiener to robust restoration From L -norm (Fourier) to L 1. (e.g., TV, Beltrami, wavelet shrinkage ) Sparsity. Sparseland From unitary to richer representations Frames, shift-invariance, steerable wavelet, contourlet, curvelet Overcompleteness. Approximation theory Non-linear approximation Sparsity & Overcompleteness. ICA and related models is HERE Independence and Sparsity. D x 1

13 To Summarize so far M The Sparseland model for signals is interesting since it is rich and yet every signal has a simple description Who cares? We do! this model is relevant to us just as well. So, what are the implications? (a) Practical solvers? (b) How will we get D? (c) Applications? Sparsity? looks complicated We need to solve (or approximate the solution of) a linear system with more unknowns than equations, while finding the sparsest possible solution D x 13

14 Agenda 1. A Visit to Sparseland Motivating Sparsity & Overcompleteness T. Problem 1: Transforms & Regularizations How & why should this work? 3. Problem : What About D? The quest for the origin of signals Q 4. Problem 3: Applications Image filling in, denoising, separation, compression, D x 14

15 Lets Start with the Transform Our dream for Now: Find the sparsest solution of D x Known Put formally, P 0 : Min 0 s.t. x D known D x 15

16 Question 1 Uniqueness? M x D Multiply by D Min s.t. 0 x D ˆ Suppose we can solve this exactly Why should we necessarily get ˆ? It might happen that eventually ˆ <. 0 0 D x 16

17 Matrix Spark Definition: Given a matrix D, σspark{d} is the smallest and and number of columns that are linearly dependent. Donoho & E. ( 0) * Example: Rank 4 Spark 3 * In tensor decomposition, Kruskal defined something similar already in D x 17

18 Uniqueness Rule Suppose this problem has been solved somehow P 0 : Min 0 s.t. x D Uniqueness Donoho & E. ( 0) If we found a representation that satisfy σ > 0 Then necessarily it is unique (the sparsest). M This result implies that if generates signals using sparse enough, the solution of the above will find it exactly. D x 18

19 Question Practical P 0 Solver? M x D Multiply by D Min s.t. 0 x D ˆ Are there reasonable ways to find ˆ? D x 19

20 Matching Pursuit (MP) Mallat & Zhang (1993) The MP is a greedy algorithm that finds one atom at a time. Step 1: find the one atom that best matches the signal. Next steps: given the previously found atoms, find the next one to best fit The Orthogonal MP (OMP) is an improved version that re-evaluates the coefficients after each round. D x 0

21 Basis Pursuit (BP) Min 0 Instead of solving s.t. x D Chen, Donoho, & Saunders (1995) Min 1 Solve Instead s.t. x D The newly defined problem is convex (linear programming). Very efficient solvers can be deployed: Interior point methods [Chen, Donoho, & Saunders (`95)], Sequential shrinkage for union of ortho-bases [Bruce et.al. (`98)], Iterated shrinkage [Figuerido & Nowak (`03), Daubechies, Defrise, & Demole ( 04), E. (`05), E., Matalon, & Zibulevsky (`06)]. D x 1

22 Question 3 Approx. Quality? M x D Multiply by D Min s.t. 0 x D ˆ How effective are the MP/BP in finding ˆ? D x

23 Evaluating the Spark Compute D D T Assume normalized columns D T D The Mutual Coherence M is the largest off-diagonal entry in absolute value. The Mutual Coherence is a property of the dictionary (just like the Spark ). In fact, the following relation can be shown: σ M D x 3

24 BP and MP Equivalence Equivalence Donoho & E. ( 0) Gribonval & Nielsen ( 03) Tropp ( 03) Temlyakov ( 03) Given a signal x with a representation x D, ( ) Assuming that < M, BP and MP are 0 Guaranteed to find the sparsest solution. MP and BP are different in general (hard to say which is better). The above result corresponds to the worst-case. Average performance results are available too, showing much better bounds [Donoho (`04), Candes et.al. (`04), Tanner et.al. (`05), Tropp et.al. (`06)]. D x 4

25 What About Inverse Problems? A sparse & random vector x Multiply by D D blur by H v y Hx + v Min 0 y HD s.t. ε ˆ We had similar questions regarding uniqueness, practical solvers, and their efficiency. It turns out that similar answers are applicable here due to several recent works [Donoho, E. and Temlyakov (`04), Tropp (`04), Fuchs (`04), Gribonval et. al. (`05)]. D x 5

26 To Summarize so far The Sparseland model for signals is relevant to us. We can design transforms and priors based on it Find the sparsest solution? Use pursuit Algorithms Why works so well? (a) How shall we find D? (b) Will this work for applications? Which? What next? A sequence of works during the past 3-4 years gives theoretic justifications for these tools behavior D x 6

27 Agenda 1. A Visit to Sparseland Motivating Sparsity & Overcompleteness. Problem 1: Transforms & Regularizations How & why should this work? 3. Problem : What About D? The quest for the origin of signals 4. Problem 3: Applications Image filling in, denoising, separation, compression, D x 7

28 Problem Setting Multiply by D M { } X P j j 1 0 L x D Given these P examples and a fixed size [N K] dictionary D: 1. Is D unique?. How would we find D? D x 8

29 Uniqueness? Comments: Uniqueness Aharon, E., & Bruckstein (`05) { x } P j j 1 If is rich enough* and if L < Spark then D is unique. M { D} Rich Enough : The signals from could be clustered to groups that share the same support. At least L+1 examples per each are needed. This result is proved constructively, but the number of examples needed to pull this off is huge we will show a far better method next. A parallel result that takes into account noise is yet to be constructed. K L D x 9

30 Practical Approach Objective X D A Min D, A P j 1 D j x Each example is a linear combination of atoms from D j s.t. j, L Each example has a sparse representation with no more than L atoms (n,k,l are assumed known, D has norm. columns) j 0 Field & Olshausen (96 ) Engan et. al. (99 ) Lewicki & Sejnowski (00 ) Cotter et. al. (03 ) Gribonval et. al. (04 ) D x 30

31 K Means For Clustering Clustering: An extreme sparse coding Initialize D D Sparse Coding Nearest Neighbor Dictionary Update Column-by-Column by Mean computation over the relevant examples X T D x 31

32 The K SVD Algorithm General Initialize D D Sparse Coding Use MP or BP Dictionary Update Column-by-Column by SVD computation X T Aharon, E., & Bruckstein (`04) D x 3

33 K SVD Sparse Coding Stage Min A P j 1 D j x j s.t. j, j 0 L D For the j th example we solve X T Min D x s.t. j 0 L Pursuit Problem!!! D x 33

34 K SVD Dictionary Update Stage D d k? { } P 1 G k : The examples in X j j and that use the column d k. The content of d k influences only the examples in G k. Let us fix all A apart from the k th column and seek both d k and the k th column to better fit the residual! D x 34

35 K SVD Dictionary Update Stage We should solve: d Min k, k T k d k E F Residual E D dk? k d k is obtained by SVD on the examples residual in G k. D x 35

36 K SVD: A Synthetic Experiment Create A 0 30 random dictionary with normalized columns Generate 000 signal examples with 3 atoms per each and add noise Train a dictionary using the KSVD and MOD and compare D D Results Relative Atoms Found MOD performance K-SVD performance Iteration D x 36

37 To Summarize so far The Sparseland model for signals is relevant to us. In order to use it effectively we need to know D How D can be found? Use the K-SVD algorithm Will it work well? (a) Deploy to applications (b) Generalize in various ways: multiscale, non-negative factorization, speed-up, (c) Performance guarantees? What next? We have established a uniqueness result and shown how to practically train D using the K-SVD D x 37

38 Agenda 1. A Visit to Sparseland Motivating Sparsity & Overcompleteness. Problem 1: Transforms & Regularizations How & why should this work? 3. Problem : What About D? The quest for the origin of signals 4. Problem 3: Applications Image filling in, denoising, separation, compression, D x 38

39 Application 1: Image Inpainting Assume: the signal x has been created by xd 0 with very sparse 0. Missing values in x imply missing rows in this linear system. By removing these rows, we get. Now solve Min 0 ~ ~ D x s.t. ~ x D ~ D 0 x If 0 was sparse enough, it will be the solution of the above problem! Thus, computing D 0 recovers x perfectly. D x 39

40 Application 1: The Practice Given a noisy image y, we can clean it using the Maximum A- posteriori Probability estimator by solving ˆ ArgMin p p + λ y D What if some of the pixels in that image are missing (filled with zeros)? Define a mask operator as the diagonal matrix W, and now solve instead p ˆ ArgMin + λ y WD p xˆ Dˆ xˆ Dˆ When handling general images, there is a need to concatenate two dictionaries to get an effective treatment of both texture and cartoon contents This leads to separation [E., Starck, & Donoho ( 05)]. D x 40

41 Inpainting Results Source Predetermined dictionary: Curvelet (cartoon) + Overlapped DCT (texture) Outcome D x 41

42 Inpainting Results Source predetermined dictionary: Curvelet (cartoon) + Overlapped DCT (texture) Outcome D x 4

43 Inpainting Results 0% 50% 80% D x 43

44 Application : Image Denoising Given a noisy image y, we have already mentioned the ability to clean it by solving ˆ ArgMin p p + λ y D When using the K-SVD, it cannot train a dictionary for large support images How do we go from local treatment of patches to a global prior? The solution: Force shift-invariant sparsity - on each patch of size N-by-N (e.g., N8) in the image, including overlaps. xˆ Dˆ D x 44

45 Application : Image Denoising ˆ ArgMin p p + λ y D xˆ Dˆ Our MAP penalty becomes xˆ ArgMin x,{ ij } ij 1 x y + μ ij R ij x D ij Extracts a patch in the ij location s.t. ij 0 0 L Our prior D x 45

46 Application : Image Denoising x,{ ij } ij, D 1 xˆ ArgMin x y + μ R ij ij x D xy and D known x and ij known ij s.t. ij 0 0 L D and ij known Compute ij per patch ij Min R s.t. ij x 0 0 D L using the matching pursuit Compute D to minimize Min R ij ijx D using SVD, updating one column at a time K-SVD Compute x by 1 T T x I + μ R ij R ij y + μ R ij Dij ij ij which is a simple averaging of shifted patches D x 46

47 Application : The Algorithm Sparse-code every patch of 8-by-8 pixels D Update the dictionary based on the above codes Compute the output image x Complexity of this algorithm: O(N L Iterations) per pixel. For N8, L1, and 10 iterations, we need 640 operations per pixel. D x 47

48 Denoising Results Source The results of this algorithm compete favorably with the state-of-the-art (e.g., GSM+steerable wavelets [Portilla, Strela, Wainwright, & Simoncelli ( 03)] - giving ~0.5-1dB better results) Result 30.89dB Noisy image σ 0 The obtained Initial dictionary after (overcomplete 10 iterations DCT) D x 48

49 Application 3: Compression The problem: Compressing photo-id images. General purpose methods (JPEG, JPEG000) do not take into account the specific family. By adapting to the image-content (PCA/K-SVD), better results could be obtained. For these techniques to operate well, train dictionaries locally (per patch) using a training set of images is required. In PCA, only the (quantized) coefficients are stored, whereas the K-SVD requires storage of the indices as well. Geometric alignment of the image is very helpful and should be done. D x 49

50 Application 3: The Algorithm Divide the image into disjoint 15-by-15 patches. For each compute mean and dictionary Per each patch find the operating parameters (number of atoms L, quantization Q) Warp, remove the mean from each patch, sparse code using L atoms, and apply Q) D x Training set (500 images) On the training set Detect main features and warp the images to a common reference (0 parameters) On the test image 50

51 Compression Results Results for 80 Bytes per each file D x 51

52 Compression Results Results for 550 Bytes per each file D x 5

53 Compression Results? ? Results for 400 Bytes per each file? D x 53

54 Today We Have Discussed 1. A Visit to Sparseland Motivating Sparsity & Overcompleteness. Problem 1: Transforms & Regularizations How & why should this work? 3. Problem : What About D? The quest for the origin of signals 4. Problem 3: Applications Image filling in, denoising, separation, compression, D x 54

55 Summary Sparsity and Overcompleteness are important ideas that can be used in designing better tools in signal/image processing There are difficulties in using them! We are working on resolving those difficulties: Performance of pursuit alg. Speedup of those methods, Training the dictionary, Demonstrating applications, Future transforms and regularizations will be datadriven, non-linear, overcomplete, and promoting sparsity. The dream? D x 55

Noise Removal? The Evolution Of Pr(x) Denoising By Energy Minimization. ( x) An Introduction to Sparse Representation and the K-SVD Algorithm

Noise Removal? The Evolution Of Pr(x) Denoising By Energy Minimization. ( x) An Introduction to Sparse Representation and the K-SVD Algorithm Sparse Representation and the K-SV Algorithm he CS epartment he echnion Israel Institute of technology Haifa 3, Israel University of Erlangen - Nürnberg April 8 Noise Removal? Our story begins with image

More information

Overcomplete Dictionaries for. Sparse Representation of Signals. Michal Aharon

Overcomplete Dictionaries for. Sparse Representation of Signals. Michal Aharon Overcomplete Dictionaries for Sparse Representation of Signals Michal Aharon ii Overcomplete Dictionaries for Sparse Representation of Signals Reasearch Thesis Submitted in Partial Fulfillment of The Requirements

More information

Sparse & Redundant Representations by Iterated-Shrinkage Algorithms

Sparse & Redundant Representations by Iterated-Shrinkage Algorithms Sparse & Redundant Representations by Michael Elad * The Computer Science Department The Technion Israel Institute of technology Haifa 3000, Israel 6-30 August 007 San Diego Convention Center San Diego,

More information

Machine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013

Machine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 Machine Learning for Signal Processing Sparse and Overcomplete Representations Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 1 Key Topics in this Lecture Basics Component-based representations

More information

ITERATED SRINKAGE ALGORITHM FOR BASIS PURSUIT MINIMIZATION

ITERATED SRINKAGE ALGORITHM FOR BASIS PURSUIT MINIMIZATION ITERATED SRINKAGE ALGORITHM FOR BASIS PURSUIT MINIMIZATION Michael Elad The Computer Science Department The Technion Israel Institute o technology Haia 3000, Israel * SIAM Conerence on Imaging Science

More information

SPARSE signal representations have gained popularity in recent

SPARSE signal representations have gained popularity in recent 6958 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 10, OCTOBER 2011 Blind Compressed Sensing Sivan Gleichman and Yonina C. Eldar, Senior Member, IEEE Abstract The fundamental principle underlying

More information

Machine Learning for Signal Processing Sparse and Overcomplete Representations

Machine Learning for Signal Processing Sparse and Overcomplete Representations Machine Learning for Signal Processing Sparse and Overcomplete Representations Abelino Jimenez (slides from Bhiksha Raj and Sourish Chaudhuri) Oct 1, 217 1 So far Weights Data Basis Data Independent ICA

More information

Sparsity in Underdetermined Systems

Sparsity in Underdetermined Systems Sparsity in Underdetermined Systems Department of Statistics Stanford University August 19, 2005 Classical Linear Regression Problem X n y p n 1 > Given predictors and response, y Xβ ε = + ε N( 0, σ 2

More information

Morphological Diversity and Source Separation

Morphological Diversity and Source Separation Morphological Diversity and Source Separation J. Bobin, Y. Moudden, J.-L. Starck, and M. Elad Abstract This paper describes a new method for blind source separation, adapted to the case of sources having

More information

Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France

Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France Inverse problems and sparse models (1/2) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Structure of the tutorial Session 1: Introduction to inverse problems & sparse

More information

EUSIPCO

EUSIPCO EUSIPCO 013 1569746769 SUBSET PURSUIT FOR ANALYSIS DICTIONARY LEARNING Ye Zhang 1,, Haolong Wang 1, Tenglong Yu 1, Wenwu Wang 1 Department of Electronic and Information Engineering, Nanchang University,

More information

Sparse molecular image representation

Sparse molecular image representation Sparse molecular image representation Sofia Karygianni a, Pascal Frossard a a Ecole Polytechnique Fédérale de Lausanne (EPFL), Signal Processing Laboratory (LTS4), CH-115, Lausanne, Switzerland Abstract

More information

Sensing systems limited by constraints: physical size, time, cost, energy

Sensing systems limited by constraints: physical size, time, cost, energy Rebecca Willett Sensing systems limited by constraints: physical size, time, cost, energy Reduce the number of measurements needed for reconstruction Higher accuracy data subject to constraints Original

More information

Signal Recovery, Uncertainty Relations, and Minkowski Dimension

Signal Recovery, Uncertainty Relations, and Minkowski Dimension Signal Recovery, Uncertainty Relations, and Minkowski Dimension Helmut Bőlcskei ETH Zurich December 2013 Joint work with C. Aubel, P. Kuppinger, G. Pope, E. Riegler, D. Stotz, and C. Studer Aim of this

More information

2.3. Clustering or vector quantization 57

2.3. Clustering or vector quantization 57 Multivariate Statistics non-negative matrix factorisation and sparse dictionary learning The PCA decomposition is by construction optimal solution to argmin A R n q,h R q p X AH 2 2 under constraint :

More information

A Generalized Uncertainty Principle and Sparse Representation in Pairs of Bases

A Generalized Uncertainty Principle and Sparse Representation in Pairs of Bases 2558 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 48, NO 9, SEPTEMBER 2002 A Generalized Uncertainty Principle Sparse Representation in Pairs of Bases Michael Elad Alfred M Bruckstein Abstract An elementary

More information

Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images

Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images Sparse Solutions of Systems of Equations and Sparse Modelling of Signals and Images Alfredo Nava-Tudela ant@umd.edu John J. Benedetto Department of Mathematics jjb@umd.edu Abstract In this project we are

More information

Sparse linear models

Sparse linear models Sparse linear models Optimization-Based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_spring16 Carlos Fernandez-Granda 2/22/2016 Introduction Linear transforms Frequency representation Short-time

More information

Five Lectures on Sparse and Redundant Representations Modelling of Images. Michael Elad

Five Lectures on Sparse and Redundant Representations Modelling of Images. Michael Elad Five Lectures on Sparse and Redundant Representations Modelling of Images Michael Elad IAS/Park City Mathematics Series Volume 19, 2010 Five Lectures on Sparse and Redundant Representations Modelling

More information

Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation

Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 19, NO. 12, DECEMBER 2008 2009 Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation Yuanqing Li, Member, IEEE, Andrzej Cichocki,

More information

The Iteration-Tuned Dictionary for Sparse Representations

The Iteration-Tuned Dictionary for Sparse Representations The Iteration-Tuned Dictionary for Sparse Representations Joaquin Zepeda #1, Christine Guillemot #2, Ewa Kijak 3 # INRIA Centre Rennes - Bretagne Atlantique Campus de Beaulieu, 35042 Rennes Cedex, FRANCE

More information

c 2011 International Press Vol. 18, No. 1, pp , March DENNIS TREDE

c 2011 International Press Vol. 18, No. 1, pp , March DENNIS TREDE METHODS AND APPLICATIONS OF ANALYSIS. c 2011 International Press Vol. 18, No. 1, pp. 105 110, March 2011 007 EXACT SUPPORT RECOVERY FOR LINEAR INVERSE PROBLEMS WITH SPARSITY CONSTRAINTS DENNIS TREDE Abstract.

More information

Tutorial: Sparse Signal Processing Part 1: Sparse Signal Representation. Pier Luigi Dragotti Imperial College London

Tutorial: Sparse Signal Processing Part 1: Sparse Signal Representation. Pier Luigi Dragotti Imperial College London Tutorial: Sparse Signal Processing Part 1: Sparse Signal Representation Pier Luigi Dragotti Imperial College London Outline Part 1: Sparse Signal Representation ~90min Part 2: Sparse Sampling ~90min 2

More information

New Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit

New Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence

More information

Recent developments on sparse representation

Recent developments on sparse representation Recent developments on sparse representation Zeng Tieyong Department of Mathematics, Hong Kong Baptist University Email: zeng@hkbu.edu.hk Hong Kong Baptist University Dec. 8, 2008 First Previous Next Last

More information

SIGNAL SEPARATION USING RE-WEIGHTED AND ADAPTIVE MORPHOLOGICAL COMPONENT ANALYSIS

SIGNAL SEPARATION USING RE-WEIGHTED AND ADAPTIVE MORPHOLOGICAL COMPONENT ANALYSIS TR-IIS-4-002 SIGNAL SEPARATION USING RE-WEIGHTED AND ADAPTIVE MORPHOLOGICAL COMPONENT ANALYSIS GUAN-JU PENG AND WEN-LIANG HWANG Feb. 24, 204 Technical Report No. TR-IIS-4-002 http://www.iis.sinica.edu.tw/page/library/techreport/tr204/tr4.html

More information

SOS Boosting of Image Denoising Algorithms

SOS Boosting of Image Denoising Algorithms SOS Boosting of Image Denoising Algorithms Yaniv Romano and Michael Elad The Technion Israel Institute of technology Haifa 32000, Israel The research leading to these results has received funding from

More information

Image Denoising with Shrinkage and Redundant Representations

Image Denoising with Shrinkage and Redundant Representations Image Denoising with Shrinkage and Redundant Representations Michael Elad Department of Computer Science The Technion - Israel Institute of Technology Haifa 32000 Israel elad@cs.technion.ac.il Michael

More information

A Quest for a Universal Model for Signals: From Sparsity to ConvNets

A Quest for a Universal Model for Signals: From Sparsity to ConvNets A Quest for a Universal Model for Signals: From Sparsity to ConvNets Yaniv Romano The Electrical Engineering Department Technion Israel Institute of technology Joint work with Vardan Papyan Jeremias Sulam

More information

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images: Final Presentation Alfredo Nava-Tudela John J. Benedetto, advisor 5/10/11 AMSC 663/664 1 Problem Let A be an n

More information

Image Noise: Detection, Measurement and Removal Techniques. Zhifei Zhang

Image Noise: Detection, Measurement and Removal Techniques. Zhifei Zhang Image Noise: Detection, Measurement and Removal Techniques Zhifei Zhang Outline Noise measurement Filter-based Block-based Wavelet-based Noise removal Spatial domain Transform domain Non-local methods

More information

An Introduction to Sparse Approximation

An Introduction to Sparse Approximation An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,

More information

A tutorial on sparse modeling. Outline:

A tutorial on sparse modeling. Outline: A tutorial on sparse modeling. Outline: 1. Why? 2. What? 3. How. 4. no really, why? Sparse modeling is a component in many state of the art signal processing and machine learning tasks. image processing

More information

EE 381V: Large Scale Optimization Fall Lecture 24 April 11

EE 381V: Large Scale Optimization Fall Lecture 24 April 11 EE 381V: Large Scale Optimization Fall 2012 Lecture 24 April 11 Lecturer: Caramanis & Sanghavi Scribe: Tao Huang 24.1 Review In past classes, we studied the problem of sparsity. Sparsity problem is that

More information

LEARNING OVERCOMPLETE SPARSIFYING TRANSFORMS FOR SIGNAL PROCESSING. Saiprasad Ravishankar and Yoram Bresler

LEARNING OVERCOMPLETE SPARSIFYING TRANSFORMS FOR SIGNAL PROCESSING. Saiprasad Ravishankar and Yoram Bresler LEARNING OVERCOMPLETE SPARSIFYING TRANSFORMS FOR SIGNAL PROCESSING Saiprasad Ravishankar and Yoram Bresler Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University

More information

Sparsity and Morphological Diversity in Source Separation. Jérôme Bobin IRFU/SEDI-Service d Astrophysique CEA Saclay - France

Sparsity and Morphological Diversity in Source Separation. Jérôme Bobin IRFU/SEDI-Service d Astrophysique CEA Saclay - France Sparsity and Morphological Diversity in Source Separation Jérôme Bobin IRFU/SEDI-Service d Astrophysique CEA Saclay - France Collaborators - Yassir Moudden - CEA Saclay, France - Jean-Luc Starck - CEA

More information

Bayesian Paradigm. Maximum A Posteriori Estimation

Bayesian Paradigm. Maximum A Posteriori Estimation Bayesian Paradigm Maximum A Posteriori Estimation Simple acquisition model noise + degradation Constraint minimization or Equivalent formulation Constraint minimization Lagrangian (unconstraint minimization)

More information

Structured matrix factorizations. Example: Eigenfaces

Structured matrix factorizations. Example: Eigenfaces Structured matrix factorizations Example: Eigenfaces An extremely large variety of interesting and important problems in machine learning can be formulated as: Given a matrix, find a matrix and a matrix

More information

The Analysis Cosparse Model for Signals and Images

The Analysis Cosparse Model for Signals and Images The Analysis Cosparse Model for Signals and Images Raja Giryes Computer Science Department, Technion. The research leading to these results has received funding from the European Research Council under

More information

Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation

Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. X, NO. X, XX 200X 1 Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation Ron Rubinstein, Student Member, IEEE, Michael Zibulevsky,

More information

Sparse Estimation and Dictionary Learning

Sparse Estimation and Dictionary Learning Sparse Estimation and Dictionary Learning (for Biostatistics?) Julien Mairal Biostatistics Seminar, UC Berkeley Julien Mairal Sparse Estimation and Dictionary Learning Methods 1/69 What this talk is about?

More information

Signal Sparsity Models: Theory and Applications

Signal Sparsity Models: Theory and Applications Signal Sparsity Models: Theory and Applications Raja Giryes Computer Science Department, Technion Michael Elad, Technion, Haifa Israel Sangnam Nam, CMI, Marseille France Remi Gribonval, INRIA, Rennes France

More information

Inverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France.

Inverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France. Inverse problems and sparse models (6/6) Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Overview of the course Introduction sparsity & data compression inverse problems

More information

Introduction to Compressed Sensing

Introduction to Compressed Sensing Introduction to Compressed Sensing Alejandro Parada, Gonzalo Arce University of Delaware August 25, 2016 Motivation: Classical Sampling 1 Motivation: Classical Sampling Issues Some applications Radar Spectral

More information

SHIFT-INVARIANT DICTIONARY LEARNING FOR SPARSE REPRESENTATIONS: EXTENDING K-SVD

SHIFT-INVARIANT DICTIONARY LEARNING FOR SPARSE REPRESENTATIONS: EXTENDING K-SVD SHIFT-INVARIANT DICTIONARY LEARNING FOR SPARSE REPRESENTATIONS: EXTENDING K-SVD Boris Mailhé, Sylvain Lesage,Rémi Gribonval and Frédéric Bimbot Projet METISS Centre de Recherche INRIA Rennes - Bretagne

More information

A simple test to check the optimality of sparse signal approximations

A simple test to check the optimality of sparse signal approximations A simple test to check the optimality of sparse signal approximations Rémi Gribonval, Rosa Maria Figueras I Ventura, Pierre Vergheynst To cite this version: Rémi Gribonval, Rosa Maria Figueras I Ventura,

More information

Compressed Sensing in Astronomy

Compressed Sensing in Astronomy Compressed Sensing in Astronomy J.-L. Starck CEA, IRFU, Service d'astrophysique, France jstarck@cea.fr http://jstarck.free.fr Collaborators: J. Bobin, CEA. Introduction: Compressed Sensing (CS) Sparse

More information

Topographic Dictionary Learning with Structured Sparsity

Topographic Dictionary Learning with Structured Sparsity Topographic Dictionary Learning with Structured Sparsity Julien Mairal 1 Rodolphe Jenatton 2 Guillaume Obozinski 2 Francis Bach 2 1 UC Berkeley 2 INRIA - SIERRA Project-Team San Diego, Wavelets and Sparsity

More information

Greedy Dictionary Selection for Sparse Representation

Greedy Dictionary Selection for Sparse Representation Greedy Dictionary Selection for Sparse Representation Volkan Cevher Rice University volkan@rice.edu Andreas Krause Caltech krausea@caltech.edu Abstract We discuss how to construct a dictionary by selecting

More information

Image representation with multi-scale gradients

Image representation with multi-scale gradients Image representation with multi-scale gradients Eero P Simoncelli Center for Neural Science, and Courant Institute of Mathematical Sciences New York University http://www.cns.nyu.edu/~eero Visual image

More information

Simultaneous Sparsity

Simultaneous Sparsity Simultaneous Sparsity Joel A. Tropp Anna C. Gilbert Martin J. Strauss {jtropp annacg martinjs}@umich.edu Department of Mathematics The University of Michigan 1 Simple Sparse Approximation Work in the d-dimensional,

More information

Large-Scale L1-Related Minimization in Compressive Sensing and Beyond

Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Large-Scale L1-Related Minimization in Compressive Sensing and Beyond Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Arizona State University March

More information

Rémi Gribonval - PANAMA Project-team INRIA Rennes - Bretagne Atlantique, France

Rémi Gribonval - PANAMA Project-team INRIA Rennes - Bretagne Atlantique, France Sparse dictionary learning in the presence of noise and outliers Rémi Gribonval - PANAMA Project-team INRIA Rennes - Bretagne Atlantique, France remi.gribonval@inria.fr Overview Context: inverse problems

More information

Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach

Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach Detecting Sparse Structures in Data in Sub-Linear Time: A group testing approach Boaz Nadler The Weizmann Institute of Science Israel Joint works with Inbal Horev, Ronen Basri, Meirav Galun and Ery Arias-Castro

More information

Randomness-in-Structured Ensembles for Compressed Sensing of Images

Randomness-in-Structured Ensembles for Compressed Sensing of Images Randomness-in-Structured Ensembles for Compressed Sensing of Images Abdolreza Abdolhosseini Moghadam Dep. of Electrical and Computer Engineering Michigan State University Email: abdolhos@msu.edu Hayder

More information

Applied Machine Learning for Biomedical Engineering. Enrico Grisan

Applied Machine Learning for Biomedical Engineering. Enrico Grisan Applied Machine Learning for Biomedical Engineering Enrico Grisan enrico.grisan@dei.unipd.it Data representation To find a representation that approximates elements of a signal class with a linear combination

More information

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images!

Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images! Sparse Solutions of Linear Systems of Equations and Sparse Modeling of Signals and Images! Alfredo Nava-Tudela John J. Benedetto, advisor 1 Happy birthday Lucía! 2 Outline - Problem: Find sparse solutions

More information

IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER

IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER 2015 1239 Preconditioning for Underdetermined Linear Systems with Sparse Solutions Evaggelia Tsiligianni, StudentMember,IEEE, Lisimachos P. Kondi,

More information

IN a series of recent papers [1] [4], the morphological component

IN a series of recent papers [1] [4], the morphological component IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 11, NOVEMBER 2007 2675 Morphological Component Analysis: An Adaptive Thresholding Strategy Jérôme Bobin, Jean-Luc Starck, Jalal M. Fadili, Yassir Moudden,

More information

Wavelet Footprints: Theory, Algorithms, and Applications

Wavelet Footprints: Theory, Algorithms, and Applications 1306 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 51, NO. 5, MAY 2003 Wavelet Footprints: Theory, Algorithms, and Applications Pier Luigi Dragotti, Member, IEEE, and Martin Vetterli, Fellow, IEEE Abstract

More information

Blind Compressed Sensing

Blind Compressed Sensing 1 Blind Compressed Sensing Sivan Gleichman and Yonina C. Eldar, Senior Member, IEEE arxiv:1002.2586v2 [cs.it] 28 Apr 2010 Abstract The fundamental principle underlying compressed sensing is that a signal,

More information

Generalized Orthogonal Matching Pursuit- A Review and Some

Generalized Orthogonal Matching Pursuit- A Review and Some Generalized Orthogonal Matching Pursuit- A Review and Some New Results Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur, INDIA Table of Contents

More information

Compressed Sensing: Extending CLEAN and NNLS

Compressed Sensing: Extending CLEAN and NNLS Compressed Sensing: Extending CLEAN and NNLS Ludwig Schwardt SKA South Africa (KAT Project) Calibration & Imaging Workshop Socorro, NM, USA 31 March 2009 Outline 1 Compressed Sensing (CS) Introduction

More information

5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER /$ IEEE

5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER /$ IEEE 5742 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 12, DECEMBER 2009 Uncertainty Relations for Shift-Invariant Analog Signals Yonina C. Eldar, Senior Member, IEEE Abstract The past several years

More information

Sparsifying Transform Learning for Compressed Sensing MRI

Sparsifying Transform Learning for Compressed Sensing MRI Sparsifying Transform Learning for Compressed Sensing MRI Saiprasad Ravishankar and Yoram Bresler Department of Electrical and Computer Engineering and Coordinated Science Laborarory University of Illinois

More information

Sparse Solutions of an Undetermined Linear System

Sparse Solutions of an Undetermined Linear System 1 Sparse Solutions of an Undetermined Linear System Maddullah Almerdasy New York University Tandon School of Engineering arxiv:1702.07096v1 [math.oc] 23 Feb 2017 Abstract This work proposes a research

More information

Sparse Modeling. in Image Processing and Deep Learning. Michael Elad

Sparse Modeling. in Image Processing and Deep Learning. Michael Elad Sparse Modeling in Image Processing and Deep Learning Computer Science Department - Israel Institute of Technology Haifa 32000, Israel The research leading to these results has been received funding from

More information

A NEW FRAMEWORK FOR DESIGNING INCOHERENT SPARSIFYING DICTIONARIES

A NEW FRAMEWORK FOR DESIGNING INCOHERENT SPARSIFYING DICTIONARIES A NEW FRAMEWORK FOR DESIGNING INCOERENT SPARSIFYING DICTIONARIES Gang Li, Zhihui Zhu, 2 uang Bai, 3 and Aihua Yu 3 School of Automation & EE, Zhejiang Univ. of Sci. & Tech., angzhou, Zhejiang, P.R. China

More information

Image compression using an edge adapted redundant dictionary and wavelets

Image compression using an edge adapted redundant dictionary and wavelets Image compression using an edge adapted redundant dictionary and wavelets Lorenzo Peotta, Lorenzo Granai, Pierre Vandergheynst Signal Processing Institute Swiss Federal Institute of Technology EPFL-STI-ITS-LTS2

More information

Compressed sensing. Or: the equation Ax = b, revisited. Terence Tao. Mahler Lecture Series. University of California, Los Angeles

Compressed sensing. Or: the equation Ax = b, revisited. Terence Tao. Mahler Lecture Series. University of California, Los Angeles Or: the equation Ax = b, revisited University of California, Los Angeles Mahler Lecture Series Acquiring signals Many types of real-world signals (e.g. sound, images, video) can be viewed as an n-dimensional

More information

Inverse problems, Dictionary based Signal Models and Compressed Sensing

Inverse problems, Dictionary based Signal Models and Compressed Sensing Inverse problems, Dictionary based Signal Models and Compressed Sensing Rémi Gribonval METISS project-team (audio signal processing, speech recognition, source separation) INRIA, Rennes, France Ecole d

More information

From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images

From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images Alfred M. Bruckstein David L. Donoho Michael Elad July 22, 2007 Abstract A full-rank matrix A IR n m with n < m generates

More information

Compressed Sensing and Neural Networks

Compressed Sensing and Neural Networks and Jan Vybíral (Charles University & Czech Technical University Prague, Czech Republic) NOMAD Summer Berlin, September 25-29, 2017 1 / 31 Outline Lasso & Introduction Notation Training the network Applications

More information

Sparse linear models and denoising

Sparse linear models and denoising Lecture notes 4 February 22, 2016 Sparse linear models and denoising 1 Introduction 1.1 Definition and motivation Finding representations of signals that allow to process them more effectively is a central

More information

Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees

Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html Acknowledgement: this slides is based on Prof. Emmanuel Candes and Prof. Wotao Yin

More information

Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery

Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan Connection between... Sparse Approximation and Compressed

More information

Analysis of Denoising by Sparse Approximation with Random Frame Asymptotics

Analysis of Denoising by Sparse Approximation with Random Frame Asymptotics Analysis of Denoising by Sparse Approximation with Random Frame Asymptotics Alyson K Fletcher Univ of California, Berkeley alyson@eecsberkeleyedu Sundeep Rangan Flarion Technologies srangan@flarioncom

More information

Morphological Diversity and Sparsity for Multichannel Data Restoration

Morphological Diversity and Sparsity for Multichannel Data Restoration DOI 10.1007/s10851-008-0065-6 Morphological Diversity and Sparsity for Multichannel Data Restoration J. Bobin Y. Moudden J. Fadili J.-L. Starck Springer Science+Business Media, LLC 2008 Abstract Over the

More information

Deterministic sampling masks and compressed sensing: Compensating for partial image loss at the pixel level

Deterministic sampling masks and compressed sensing: Compensating for partial image loss at the pixel level Deterministic sampling masks and compressed sensing: Compensating for partial image loss at the pixel level Alfredo Nava-Tudela Institute for Physical Science and Technology and Norbert Wiener Center,

More information

On the Projection Matrices Influence in the Classification of Compressed Sensed ECG Signals

On the Projection Matrices Influence in the Classification of Compressed Sensed ECG Signals On the Projection Matrices Influence in the Classification of Compressed Sensed ECG Signals Monica Fira, Liviu Goras Institute of Computer Science Romanian Academy Iasi, Romania Liviu Goras, Nicolae Cleju,

More information

Introduction to Sparsity. Xudong Cao, Jake Dreamtree & Jerry 04/05/2012

Introduction to Sparsity. Xudong Cao, Jake Dreamtree & Jerry 04/05/2012 Introduction to Sparsity Xudong Cao, Jake Dreamtree & Jerry 04/05/2012 Outline Understanding Sparsity Total variation Compressed sensing(definition) Exact recovery with sparse prior(l 0 ) l 1 relaxation

More information

A new method on deterministic construction of the measurement matrix in compressed sensing

A new method on deterministic construction of the measurement matrix in compressed sensing A new method on deterministic construction of the measurement matrix in compressed sensing Qun Mo 1 arxiv:1503.01250v1 [cs.it] 4 Mar 2015 Abstract Construction on the measurement matrix A is a central

More information

A simple test to check the optimality of sparse signal approximations

A simple test to check the optimality of sparse signal approximations A simple test to check the optimality of sparse signal approximations Rémi Gribonval, Rosa Maria Figueras I Ventura, Pierre Vandergheynst To cite this version: Rémi Gribonval, Rosa Maria Figueras I Ventura,

More information

Bayesian Methods for Sparse Signal Recovery

Bayesian Methods for Sparse Signal Recovery Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Jason Palmer, Zhilin Zhang and Ritwik Giri Motivation Motivation Sparse Signal Recovery

More information

Efficient Data-Driven Learning of Sparse Signal Models and Its Applications

Efficient Data-Driven Learning of Sparse Signal Models and Its Applications Efficient Data-Driven Learning of Sparse Signal Models and Its Applications Saiprasad Ravishankar Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor Dec 10, 2015

More information

MATCHING PURSUIT WITH STOCHASTIC SELECTION

MATCHING PURSUIT WITH STOCHASTIC SELECTION 2th European Signal Processing Conference (EUSIPCO 22) Bucharest, Romania, August 27-3, 22 MATCHING PURSUIT WITH STOCHASTIC SELECTION Thomas Peel, Valentin Emiya, Liva Ralaivola Aix-Marseille Université

More information

Scalable color image coding with Matching Pursuit

Scalable color image coding with Matching Pursuit SCHOOL OF ENGINEERING - STI SIGNAL PROCESSING INSTITUTE Rosa M. Figueras i Ventura CH-115 LAUSANNE Telephone: +4121 6935646 Telefax: +4121 69376 e-mail: rosa.figueras@epfl.ch ÉCOLE POLYTECHNIQUE FÉDÉRALE

More information

Near Optimal Signal Recovery from Random Projections

Near Optimal Signal Recovery from Random Projections 1 Near Optimal Signal Recovery from Random Projections Emmanuel Candès, California Institute of Technology Multiscale Geometric Analysis in High Dimensions: Workshop # 2 IPAM, UCLA, October 2004 Collaborators:

More information

Compressed Sensing. 1 Introduction. 2 Design of Measurement Matrices

Compressed Sensing. 1 Introduction. 2 Design of Measurement Matrices Compressed Sensing Yonina C. Eldar Electrical Engineering Department, Technion-Israel Institute of Technology, Haifa, Israel, 32000 1 Introduction Compressed sensing (CS) is an exciting, rapidly growing

More information

STRUCTURE-AWARE DICTIONARY LEARNING WITH HARMONIC ATOMS

STRUCTURE-AWARE DICTIONARY LEARNING WITH HARMONIC ATOMS 19th European Signal Processing Conference (EUSIPCO 2011) Barcelona, Spain, August 29 - September 2, 2011 STRUCTURE-AWARE DICTIONARY LEARNING WITH HARMONIC ATOMS Ken O Hanlon and Mark D.Plumbley Queen

More information

Dictionary learning for speech based on a doubly sparse greedy adaptive dictionary algorithm

Dictionary learning for speech based on a doubly sparse greedy adaptive dictionary algorithm IEEE TRANSACTIONS ON, VOL. X, NO. X, JANUARY XX 1 Dictionary learning for speech based on a doubly sparse greedy adaptive dictionary algorithm Maria G. Jafari and Mark D. Plumbley Abstract In this paper

More information

Dictionary Learning for L1-Exact Sparse Coding

Dictionary Learning for L1-Exact Sparse Coding Dictionary Learning for L1-Exact Sparse Coding Mar D. Plumbley Department of Electronic Engineering, Queen Mary University of London, Mile End Road, London E1 4NS, United Kingdom. Email: mar.plumbley@elec.qmul.ac.u

More information

of Orthogonal Matching Pursuit

of Orthogonal Matching Pursuit A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit Qun Mo arxiv:50.0708v [cs.it] 8 Jan 205 Abstract We shall show that if the restricted isometry constant (RIC) δ s+ (A) of the measurement

More information

Directionlets. Anisotropic Multi-directional Representation of Images with Separable Filtering. Vladan Velisavljević Deutsche Telekom, Laboratories

Directionlets. Anisotropic Multi-directional Representation of Images with Separable Filtering. Vladan Velisavljević Deutsche Telekom, Laboratories Directionlets Anisotropic Multi-directional Representation of Images with Separable Filtering Vladan Velisavljević Deutsche Telekom, Laboratories Google Inc. Mountain View, CA October 2006 Collaborators

More information

Image Processing by the Curvelet Transform

Image Processing by the Curvelet Transform Image Processing by the Curvelet Transform Jean Luc Starck Dapnia/SEDI SAP, CEA Saclay, France. jstarck@cea.fr http://jstarck.free.fr Collaborators: D.L. Donoho, Department of Statistics, Stanford E. Candès,

More information

Uncertainty principles and sparse approximation

Uncertainty principles and sparse approximation Uncertainty principles and sparse approximation In this lecture, we will consider the special case where the dictionary Ψ is composed of a pair of orthobases. We will see that our ability to find a sparse

More information

17 Random Projections and Orthogonal Matching Pursuit

17 Random Projections and Orthogonal Matching Pursuit 17 Random Projections and Orthogonal Matching Pursuit Again we will consider high-dimensional data P. Now we will consider the uses and effects of randomness. We will use it to simplify P (put it in a

More information

Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit

Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit Claremont Colleges Scholarship @ Claremont CMC Faculty Publications and Research CMC Faculty Scholarship 6-5-2008 Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit

More information

Uniqueness Conditions for A Class of l 0 -Minimization Problems

Uniqueness Conditions for A Class of l 0 -Minimization Problems Uniqueness Conditions for A Class of l 0 -Minimization Problems Chunlei Xu and Yun-Bin Zhao October, 03, Revised January 04 Abstract. We consider a class of l 0 -minimization problems, which is to search

More information

Mathematical introduction to Compressed Sensing

Mathematical introduction to Compressed Sensing Mathematical introduction to Compressed Sensing Lesson 1 : measurements and sparsity Guillaume Lecué ENSAE Mardi 31 janvier 2016 Guillaume Lecué (ENSAE) Compressed Sensing Mardi 31 janvier 2016 1 / 31

More information