Sparse Interactions: Identifying High-Dimensional Multilinear Systems via Compressed Sensing
|
|
- Annabel Sharp
- 6 years ago
- Views:
Transcription
1 Sparse Interactions: Identifying High-Dimensional Multilinear Systems via Compressed Sensing Bobak Nazer and Robert D. Nowak University of Wisconsin, Madison Allerton 10/01/10
2 Motivation: Virus-Host Interaction virus fruit fly from Paul Ahlquist and Yoshihiro Kawaoka s Labs at UW - Madison.
3 Motivation: Virus-Host Interaction virus c Add dsrna of the Drosophila RNAi library (targeting to 13,071 Drosophila genes) to each well of 384-well microplates 0 h Add DL1 cells to the plates fruit fly GFP sensors 48 h Infect with FVG-R virus 72 h Measure Renilla luciferase activity to assess the efficiency of virus replication from Paul Ahlquist and Yoshihiro Kawaoka s Labs at UW - Madison.
4 Motivation: Virus-Host Interaction virus c Add dsrna of the Drosophila RNAi library (targeting to 13,071 Drosophila genes) to each well of 384-well microplates 0 h Add DL1 cells to the plates fruit fly GFP sensors 48 h Infect with FVG-R virus 72 h Measure Renilla luciferase activity to assess the efficiency of virus replication from Paul Ahlquist and Yoshihiro Kawaoka s Labs at UW - Madison. Which genes does the virus hijack for reproduction?
5 Basic Model Genome Hijacked Proteins Discovered Hijacked Genes
6 Basic Model Genome Knockdown Hijacked Proteins Discovered Hijacked Genes
7 Basic Model Genome Knockdown Hijacked Proteins Discovered Hijacked Genes
8 Basic Model Genome Knockdown Hijacked Proteins Discovered Hijacked Genes
9 Basic Model Genome Knockdown Hijacked Proteins Discovered Hijacked Genes
10 Basic Model Genome Knockdown Hijacked Proteins Discovered Hijacked Genes 13,071 genes to check in a fruit fly.
11 Redundant Model Genome Hijacked Proteins Discovered Hijacked Genes
12 Redundant Model Genome Knockdown Hijacked Proteins Discovered Hijacked Genes
13 Redundant Model Genome Knockdown Knockdown Hijacked Proteins Discovered Hijacked Genes
14 Redundant Model Genome Knockdown Knockdown Hijacked Proteins Discovered Hijacked Genes
15 Redundant Model Genome Knockdown Hijacked Proteins Discovered Hijacked Genes
16 Redundant Model Genome Knockdown Knockdown Hijacked Proteins Discovered Hijacked Genes
17 Redundant Model Genome Knockdown Knockdown Hijacked Proteins Discovered Hijacked Genes Millions of pairs of genes to check in a fruit fly.
18 Sparse Linear Systems Genome Knockdown y = i a i x i Can model output (virus expression) as a sparse linear system. x i a i corresponds to whether gene is involved or not. corresponds to knockdowns.
19 Sparse Multilinear Systems Genome Knockdown Knockdown y = i a i x i + i<j a i a j x ij Can model output (virus expression) as a sparse multilinear system. x i a i corresponds to whether gene is involved or not. corresponds to knockdowns.
20 Sparse Multilinear Systems Genome Knockdown y = i a i x i + i<j a i a j x ij Can model output (virus expression) as a sparse multilinear system. x i a i corresponds to whether gene is involved or not. corresponds to knockdowns.
21 This Talk How many measurements are needed to infer the coefficients for a sparse multilinear system? y = i 1 a i1 x i1 + i 1 <i 2 a i1 a i2 x i1 i i 1 <i 2 < <i D a i1 a i2 a id x i1 i 2 i D
22 This Talk How many measurements are needed to infer the coefficients for a sparse multilinear system? y = i 1 a i1 x i1 + i 1 <i 2 a i1 a i2 x i1 i i 1 <i 2 < <i D a i1 a i2 a id x i1 i 2 i D Sufficient to consider order D interactions y = i 1 <i 2 < <i D a i1 a i2 a id x i1 i 2 i D
23 Vectorized Multilinear System y k = i 1 <i 2 < <i D a ki1 a ki2 a kid x i1 i 2 i D ( Take inputs to be binary symmetric for simplicity. )
24 Vectorized Multilinear System y k = i 1 <i 2 < <i D a ki1 a ki2 a kid x i1 i 2 i D ( Take inputs to be binary symmetric for simplicity. ) Write coefficients as a vector {x i1 i 2 i } x D
25 Vectorized Multilinear System y k = i 1 <i 2 < <i D a ki1 a ki2 a kid x i1 i 2 i D ( Take inputs to be binary symmetric for simplicity. ) Write coefficients as a vector {x i1 i 2 i } x D Write measurements as a vector {a ki1 a ki2 a kid } a k
26 Vectorized Multilinear System y k = i 1 <i 2 < <i D a ki1 a ki2 a kid x i1 i 2 i D ( Take inputs to be binary symmetric for simplicity. ) Write coefficients as a vector {x i1 i 2 i } x D Write measurements as a vector {a ki1 a ki2 a kid } a k Back to a linear problem: y = Ax with potentially dependent measurements.
27 Vectorized Multilinear System y k = i 1 <i 2 < <i D a ki1 a ki2 a kid x i1 i 2 i D ( Take inputs to be binary symmetric for simplicity. ) Write coefficients as a vector {x i1 i 2 i } x D Write measurements as a vector {a ki1 a ki2 a kid } a k Back to a linear problem: y = Ax with potentially dependent measurements. Can we get down to S log (N/S) measurements?
28 Compressed Sensing Candes-Romberg-Tao ʼ06, Candes-Tao ʼ06, Donoho ʼ06 Linear measurements: Coefficients are S-sparse: y k = i y = Ax a ki x i = a T k x x 0 = S x R N
29 Compressed Sensing Candes-Romberg-Tao ʼ06, Candes-Tao ʼ06, Donoho ʼ06 Linear measurements: Coefficients are S-sparse: y k = i y = Ax a ki x i = a T k x x 0 = S x R N If the measurement matrix satisfies the restricted isometry property (RIP) with δ 2S < 2 1 for all S-sparse vectors: (1 δ S )x 2 2 Ax 2 2 (1 + δ S )x 2 2
30 Compressed Sensing Candes-Romberg-Tao ʼ06, Candes-Tao ʼ06, Donoho ʼ06 Linear measurements: Coefficients are S-sparse: y k = i y = Ax a ki x i = a T k x x 0 = S x R N If the measurement matrix satisfies the restricted isometry property (RIP) with δ 2S < 2 1 for all S-sparse vectors: (1 δ S )x 2 2 Ax 2 2 (1 + δ S )x 2 2 Then we can recover the coefficients through an 1 optimization: min ˆx 1 subject to Aˆx = y
31 Compressed Sensing Candes-Romberg-Tao ʼ06, Candes-Tao ʼ06, Donoho ʼ06 Linear measurements: Coefficients are S-sparse: y k = i y = Ax a ki x i = a T k x x 0 = S x R N If the measurement matrix satisfies the restricted isometry property (RIP) with δ 2S < 2 1 for all S-sparse vectors: (1 δ S )x 2 2 Ax 2 2 (1 + δ S )x 2 2 Then we can recover the coefficients through an 1 optimization: min ˆx 1 subject to Aˆx = y Can get RIP with K cs log(n/s) measurements.
32 Norm Preservation The restricted isometry property guarantees that the norms of all sparse vectors are approximately preserved. Example: M = 3 inputs, D = 2 interactions a =[a 1 a 2 a 1 a 3 a 2 a 3 ]
33 Norm Preservation The restricted isometry property guarantees that the norms of all sparse vectors are approximately preserved. Example: M = 3 inputs, D = 2 interactions a =[a 1 a 2 a 1 a 3 a 2 a 3 ] Our measurements preserve the norm in expectation: E y 2 = x T E A T A x Each diagonal element is the sum of products of squared terms. Each off-diagonal element is the sum of products, with at least one unique term. E A T A = I E y 2 = x 2
34 Example: Best Case Sparsity Pattern If each index is involved in at most one interaction, then the measurements concentrate very quickly. Example: y = a 1 a 2 x 12 + a 3 a 5 x 35 + a 4 a 7 x 47
35 Example: Best Case Sparsity Pattern If each index is involved in at most one interaction, then the measurements concentrate very quickly. Example: y = a 1 a 2 x 12 + a 3 a 5 x 35 + a 4 a 7 x 47 Relabel y =ã 1 x 1 +ã 2 x 2 +ã 3 x 3
36 Example: Best Case Sparsity Pattern If each index is involved in at most one interaction, then the measurements concentrate very quickly. Example: y = a 1 a 2 x 12 + a 3 a 5 x 35 + a 4 a 7 x 47 Relabel y =ã 1 x 1 +ã 2 x 2 +ã 3 x 3 Best case concentration is subgaussian: inf T P( y2 k 1 >t) exp( ct)
37 Example: Worst Case Sparsity Pattern If each a subset of indices is involved in event interaction, then the measurements concentrate more slowly. Example: y = a i a j P y 2 S 2 S 1 i j S Worst case concentration is subexponential: sup T P( y 2 k 1 >t) exp( ct 1/D )
38 Eigenvalue Experiment Look at minimum eigenvalues of subsets of Gram matrix for order D = 1, 2, 3 interactions. Minimum Eigenvalue D=1 D=2 D= Size Obviously, there is some cost incurred by dependencies within the vector.
39 Gershgorinʼs Disc Theorem Approach inspired by Haupt-Bajwa-Raz-Nowak 08: i) Control each element of the Gram matrix G R = A T RA R via Hoeffding s inequality. Bound probability that Gram matrix is at worst: δ 1 SS δ SS δ SS δ 1 SS G R = δ SS δ SS 1
40 Gershgorinʼs Disc Theorem Approach inspired by Haupt-Bajwa-Raz-Nowak 08: i) Control each element of the Gram matrix G R = A T RA R via Hoeffding s inequality. Bound probability that Gram matrix is at worst: δ 1 SS δ SS δ SS δ 1 SS G R = δ SS δ SS 1 ii) Gershgorin s Disc Theorem guarantees that eigenvalues lie in the range: g ii j=i g ij λ i (G R ) g ii + j=i g ij
41 Gershgorinʼs Disc Theorem Approach inspired by Haupt-Bajwa-Raz-Nowak 08: i) Control each element of the Gram matrix G R = A T RA R via Hoeffding s inequality. Bound probability that Gram matrix is at worst: δ 1 SS δ SS δ SS δ 1 SS G R = δ SS δ SS 1 ii) Gershgorin s Disc Theorem guarantees that eigenvalues lie in the range: g ii j=i g ij λ i (G R ) g ii + j=i g ij N iii) Union bound over all S sparse patterns. Get RIP with constant long as number of measurements is at least: δ S K = cs 2 log N
42 Heavy-Tailed Restricted Isometries Rudelson-Vershynin 08, Vershynin 10: Assume the expected Gram matrix is identity,, and the rows of the measurement matrix are independent, then if the number of measurements is at least E A T A = I K = c S 2 log S 2 3 log N with expected RIP constant E[δ S ]
43 Heavy-Tailed Restricted Isometries Rudelson-Vershynin 08, Vershynin 10: Assume the expected Gram matrix is identity,, and the rows of the measurement matrix are independent, then if the number of measurements is at least E A T A = I K = c S 2 log S 2 3 log N with expected RIP constant E[δ S ] One of the key points in the proof is that it avoids using a union bound and insteads bounds the RIP constants of all sparsity patterns simultaneously.
44 Heavy-Tailed Restricted Isometries Rudelson-Vershynin 08, Vershynin 10: Assume the expected Gram matrix is identity,, and the rows of the measurement matrix are independent, then if the number of measurements is at least E A T A = I K = c S 2 log S 2 3 log N with expected RIP constant E[δ S ] One of the key points in the proof is that it avoids using a union bound and insteads bounds the RIP constants of all sparsity patterns simultaneously. This corresponds exactly to our linearized problem.
45 Rademacher Chaos of order D: How exactly do the tails behave? y = i 1 <i 2 < <i D a i1 a i2 a id x i1 i 2 i D a i are independent and binary symmetric (Rademacher).
46 Rademacher Chaos of order D: How exactly do the tails behave? y = i 1 <i 2 < <i D a i1 a i2 a id x i1 i 2 i D a i are independent and binary symmetric (Rademacher). The combinatorial dimension measures the level of dependence introduced by the sparsity pattern. A chaos has combinatorial dimension if T L T L (A 1 A 2... A D ) sup A 1,A 2,...,A (max D 1 j D A j ) α C 1 α T L C 2 L α Takes values between 1 α D
47 Chaos Tail Bounds Blei-Janson 04: A Rademacher chaos with combinatorial dimension satisfies: exp c 1 t 2/α sup L P ( u >t) exp c 2 t 2/α α
48 Chaos Tail Bounds Blei-Janson 04: A Rademacher chaos with combinatorial dimension satisfies: exp c 1 t 2/α sup L P ( u >t) exp c 2 t 2/α α Using this bound it can be shown that: P y 2 1 exp c max Kt/S, K 1/α t 1/α from which we can derive the number of measurements needed to get RIP for a single pattern.
49 Chaos Tail Bounds Blei-Janson 04: A Rademacher chaos with combinatorial dimension satisfies: exp c 1 t 2/α sup L P ( u >t) exp c 2 t 2/α α Using this bound it can be shown that: P y 2 1 exp c max Kt/S, K 1/α t 1/α from which we can derive the number of measurements needed to get RIP for a single pattern. Union bound technique from Baraniuk-Devore-Davenport-Wakin 08: K min cs 2 log(n/s), c S α log α (N/S)
50 Summary of Bounds Gershgorin S 2 log N Second Moment Union Bound Rudelson-Vershynin S(log 3 S)(log N) Second Moment No Union Bound Rademacher Chaos S α log α (N/S) Tail Bounds Union Bound
51 Conclusions and Future Directions Can recover sparse vectors from multilinear systems despite heavytailed behavior and dependencies. Number of measurements may depend on the pattern of sparsity. What is possible for polynomial systems?
52 Acknowledgments Thanks to Paul Ahlquist for introducing us to virus replication. Thanks to Ben Recht for introducing us to the work of Rudelson and Vershynin.
Large-Scale Multiple Testing in Medical Informatics
Large-Scale Multiple Testing in Medical Informatics experiment space model space data IMA Workshop on Large Data Sets in Medical Informatics November 16 2011 Rob Nowak www.ece.wisc.edu/~nowak Joint work
More informationCS 229r: Algorithms for Big Data Fall Lecture 19 Nov 5
CS 229r: Algorithms for Big Data Fall 215 Prof. Jelani Nelson Lecture 19 Nov 5 Scribe: Abdul Wasay 1 Overview In the last lecture, we started discussing the problem of compressed sensing where we are given
More informationNew Coherence and RIP Analysis for Weak. Orthogonal Matching Pursuit
New Coherence and RIP Analysis for Wea 1 Orthogonal Matching Pursuit Mingrui Yang, Member, IEEE, and Fran de Hoog arxiv:1405.3354v1 [cs.it] 14 May 2014 Abstract In this paper we define a new coherence
More informationCompressed Sensing: Lecture I. Ronald DeVore
Compressed Sensing: Lecture I Ronald DeVore Motivation Compressed Sensing is a new paradigm for signal/image/function acquisition Motivation Compressed Sensing is a new paradigm for signal/image/function
More informationReconstruction from Anisotropic Random Measurements
Reconstruction from Anisotropic Random Measurements Mark Rudelson and Shuheng Zhou The University of Michigan, Ann Arbor Coding, Complexity, and Sparsity Workshop, 013 Ann Arbor, Michigan August 7, 013
More informationModel-Based Compressive Sensing for Signal Ensembles. Marco F. Duarte Volkan Cevher Richard G. Baraniuk
Model-Based Compressive Sensing for Signal Ensembles Marco F. Duarte Volkan Cevher Richard G. Baraniuk Concise Signal Structure Sparse signal: only K out of N coordinates nonzero model: union of K-dimensional
More informationCompressive Sensing with Random Matrices
Compressive Sensing with Random Matrices Lucas Connell University of Georgia 9 November 017 Lucas Connell (University of Georgia) Compressive Sensing with Random Matrices 9 November 017 1 / 18 Overview
More informationZ Algorithmic Superpower Randomization October 15th, Lecture 12
15.859-Z Algorithmic Superpower Randomization October 15th, 014 Lecture 1 Lecturer: Bernhard Haeupler Scribe: Goran Žužić Today s lecture is about finding sparse solutions to linear systems. The problem
More informationStrengthened Sobolev inequalities for a random subspace of functions
Strengthened Sobolev inequalities for a random subspace of functions Rachel Ward University of Texas at Austin April 2013 2 Discrete Sobolev inequalities Proposition (Sobolev inequality for discrete images)
More informationRecovering overcomplete sparse representations from structured sensing
Recovering overcomplete sparse representations from structured sensing Deanna Needell Claremont McKenna College Feb. 2015 Support: Alfred P. Sloan Foundation and NSF CAREER #1348721. Joint work with Felix
More informationSparse Optimization Lecture: Sparse Recovery Guarantees
Those who complete this lecture will know Sparse Optimization Lecture: Sparse Recovery Guarantees Sparse Optimization Lecture: Sparse Recovery Guarantees Instructor: Wotao Yin Department of Mathematics,
More informationEnhanced Compressive Sensing and More
Enhanced Compressive Sensing and More Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Nonlinear Approximation Techniques Using L1 Texas A & M University
More informationSupremum of simple stochastic processes
Subspace embeddings Daniel Hsu COMS 4772 1 Supremum of simple stochastic processes 2 Recap: JL lemma JL lemma. For any ε (0, 1/2), point set S R d of cardinality 16 ln n S = n, and k N such that k, there
More informationMAT 585: Johnson-Lindenstrauss, Group testing, and Compressed Sensing
MAT 585: Johnson-Lindenstrauss, Group testing, and Compressed Sensing Afonso S. Bandeira April 9, 2015 1 The Johnson-Lindenstrauss Lemma Suppose one has n points, X = {x 1,..., x n }, in R d with d very
More informationGeometry of log-concave Ensembles of random matrices
Geometry of log-concave Ensembles of random matrices Nicole Tomczak-Jaegermann Joint work with Radosław Adamczak, Rafał Latała, Alexander Litvak, Alain Pajor Cortona, June 2011 Nicole Tomczak-Jaegermann
More informationLecture: Introduction to Compressed Sensing Sparse Recovery Guarantees
Lecture: Introduction to Compressed Sensing Sparse Recovery Guarantees http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html Acknowledgement: this slides is based on Prof. Emmanuel Candes and Prof. Wotao Yin
More informationInterpolation via weighted l 1 minimization
Interpolation via weighted l 1 minimization Rachel Ward University of Texas at Austin December 12, 2014 Joint work with Holger Rauhut (Aachen University) Function interpolation Given a function f : D C
More informationCombining geometry and combinatorics
Combining geometry and combinatorics A unified approach to sparse signal recovery Anna C. Gilbert University of Michigan joint work with R. Berinde (MIT), P. Indyk (MIT), H. Karloff (AT&T), M. Strauss
More informationCompressed Sensing and Sparse Recovery
ELE 538B: Sparsity, Structure and Inference Compressed Sensing and Sparse Recovery Yuxin Chen Princeton University, Spring 217 Outline Restricted isometry property (RIP) A RIPless theory Compressed sensing
More informationAdaptive Information and Optimization
Adaptive Information and Optimization experiment space model space data WIDDOW 31 January 2011 Rob Nowak www.ece.wisc.edu/~nowak Joint work with R. Castro, J. Haupt, M. Malloy Adaptive Information Goal:
More informationSparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery
Sparse analysis Lecture VII: Combining geometry and combinatorics, sparse matrices for sparse signal recovery Anna C. Gilbert Department of Mathematics University of Michigan Sparse signal recovery measurements:
More informationAn Introduction to Sparse Approximation
An Introduction to Sparse Approximation Anna C. Gilbert Department of Mathematics University of Michigan Basic image/signal/data compression: transform coding Approximate signals sparsely Compress images,
More informationExponential decay of reconstruction error from binary measurements of sparse signals
Exponential decay of reconstruction error from binary measurements of sparse signals Deanna Needell Joint work with R. Baraniuk, S. Foucart, Y. Plan, and M. Wootters Outline Introduction Mathematical Formulation
More informationDimensionality Reduction Notes 3
Dimensionality Reduction Notes 3 Jelani Nelson minilek@seas.harvard.edu August 13, 2015 1 Gordon s theorem Let T be a finite subset of some normed vector space with norm X. We say that a sequence T 0 T
More informationSparse Recovery with Pre-Gaussian Random Matrices
Sparse Recovery with Pre-Gaussian Random Matrices Simon Foucart Laboratoire Jacques-Louis Lions Université Pierre et Marie Curie Paris, 75013, France Ming-Jun Lai Department of Mathematics University of
More informationLecture 13 October 6, Covering Numbers and Maurey s Empirical Method
CS 395T: Sublinear Algorithms Fall 2016 Prof. Eric Price Lecture 13 October 6, 2016 Scribe: Kiyeon Jeon and Loc Hoang 1 Overview In the last lecture we covered the lower bound for p th moment (p > 2) and
More informationCompressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery
Compressibility of Infinite Sequences and its Interplay with Compressed Sensing Recovery Jorge F. Silva and Eduardo Pavez Department of Electrical Engineering Information and Decision Systems Group Universidad
More informationINDUSTRIAL MATHEMATICS INSTITUTE. B.S. Kashin and V.N. Temlyakov. IMI Preprint Series. Department of Mathematics University of South Carolina
INDUSTRIAL MATHEMATICS INSTITUTE 2007:08 A remark on compressed sensing B.S. Kashin and V.N. Temlyakov IMI Preprint Series Department of Mathematics University of South Carolina A remark on compressed
More informationSparse Legendre expansions via l 1 minimization
Sparse Legendre expansions via l 1 minimization Rachel Ward, Courant Institute, NYU Joint work with Holger Rauhut, Hausdorff Center for Mathematics, Bonn, Germany. June 8, 2010 Outline Sparse recovery
More information6 Compressed Sensing and Sparse Recovery
6 Compressed Sensing and Sparse Recovery Most of us have noticed how saving an image in JPEG dramatically reduces the space it occupies in our hard drives as oppose to file types that save the pixel value
More informationConstructing Explicit RIP Matrices and the Square-Root Bottleneck
Constructing Explicit RIP Matrices and the Square-Root Bottleneck Ryan Cinoman July 18, 2018 Ryan Cinoman Constructing Explicit RIP Matrices July 18, 2018 1 / 36 Outline 1 Introduction 2 Restricted Isometry
More informationCS on CS: Computer Science insights into Compresive Sensing (and vice versa) Piotr Indyk MIT
CS on CS: Computer Science insights into Compresive Sensing (and vice versa) Piotr Indyk MIT Sparse Approximations Goal: approximate a highdimensional vector x by x that is sparse, i.e., has few nonzero
More informationAN INTRODUCTION TO COMPRESSIVE SENSING
AN INTRODUCTION TO COMPRESSIVE SENSING Rodrigo B. Platte School of Mathematical and Statistical Sciences APM/EEE598 Reverse Engineering of Complex Dynamical Networks OUTLINE 1 INTRODUCTION 2 INCOHERENCE
More informationIEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER
IEEE SIGNAL PROCESSING LETTERS, VOL. 22, NO. 9, SEPTEMBER 2015 1239 Preconditioning for Underdetermined Linear Systems with Sparse Solutions Evaggelia Tsiligianni, StudentMember,IEEE, Lisimachos P. Kondi,
More informationLecture Notes 9: Constrained Optimization
Optimization-based data analysis Fall 017 Lecture Notes 9: Constrained Optimization 1 Compressed sensing 1.1 Underdetermined linear inverse problems Linear inverse problems model measurements of the form
More informationA Generalized Restricted Isometry Property
1 A Generalized Restricted Isometry Property Jarvis Haupt and Robert Nowak Department of Electrical and Computer Engineering, University of Wisconsin Madison University of Wisconsin Technical Report ECE-07-1
More informationIntroduction How it works Theory behind Compressed Sensing. Compressed Sensing. Huichao Xue. CS3750 Fall 2011
Compressed Sensing Huichao Xue CS3750 Fall 2011 Table of Contents Introduction From News Reports Abstract Definition How it works A review of L 1 norm The Algorithm Backgrounds for underdetermined linear
More informationRandom Coding for Fast Forward Modeling
Random Coding for Fast Forward Modeling Justin Romberg with William Mantzel, Salman Asif, Karim Sabra, Ramesh Neelamani Georgia Tech, School of ECE Workshop on Sparsity and Computation June 11, 2010 Bonn,
More informationSolution Recovery via L1 minimization: What are possible and Why?
Solution Recovery via L1 minimization: What are possible and Why? Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, U.S.A. Eighth US-Mexico Workshop on Optimization
More informationPerformance Analysis for Sparse Support Recovery
Performance Analysis for Sparse Support Recovery Gongguo Tang and Arye Nehorai ESE, Washington University April 21st 2009 Gongguo Tang and Arye Nehorai (Institute) Performance Analysis for Sparse Support
More informationShifting Inequality and Recovery of Sparse Signals
Shifting Inequality and Recovery of Sparse Signals T. Tony Cai Lie Wang and Guangwu Xu Astract In this paper we present a concise and coherent analysis of the constrained l 1 minimization method for stale
More informationA Power Efficient Sensing/Communication Scheme: Joint Source-Channel-Network Coding by Using Compressive Sensing
Forty-Ninth Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 28-30, 20 A Power Efficient Sensing/Communication Scheme: Joint Source-Channel-Network Coding by Using Compressive Sensing
More informationThe uniform uncertainty principle and compressed sensing Harmonic analysis and related topics, Seville December 5, 2008
The uniform uncertainty principle and compressed sensing Harmonic analysis and related topics, Seville December 5, 2008 Emmanuel Candés (Caltech), Terence Tao (UCLA) 1 Uncertainty principles A basic principle
More informationMultipath Matching Pursuit
Multipath Matching Pursuit Submitted to IEEE trans. on Information theory Authors: S. Kwon, J. Wang, and B. Shim Presenter: Hwanchol Jang Multipath is investigated rather than a single path for a greedy
More informationRandom projections. 1 Introduction. 2 Dimensionality reduction. Lecture notes 5 February 29, 2016
Lecture notes 5 February 9, 016 1 Introduction Random projections Random projections are a useful tool in the analysis and processing of high-dimensional data. We will analyze two applications that use
More informationInvertibility of random matrices
University of Michigan February 2011, Princeton University Origins of Random Matrix Theory Statistics (Wishart matrices) PCA of a multivariate Gaussian distribution. [Gaël Varoquaux s blog gael-varoquaux.info]
More informationCompressive Sensing and Beyond
Compressive Sensing and Beyond Sohail Bahmani Gerorgia Tech. Signal Processing Compressed Sensing Signal Models Classics: bandlimited The Sampling Theorem Any signal with bandwidth B can be recovered
More informationGoing off the grid. Benjamin Recht Department of Computer Sciences University of Wisconsin-Madison
Going off the grid Benjamin Recht Department of Computer Sciences University of Wisconsin-Madison Joint work with Badri Bhaskar Parikshit Shah Gonnguo Tang We live in a continuous world... But we work
More informationCompressive Sensing Theory and L1-Related Optimization Algorithms
Compressive Sensing Theory and L1-Related Optimization Algorithms Yin Zhang Department of Computational and Applied Mathematics Rice University, Houston, Texas, USA CAAM Colloquium January 26, 2009 Outline:
More informationLinear Regression with Strongly Correlated Designs Using Ordered Weigthed l 1
Linear Regression with Strongly Correlated Designs Using Ordered Weigthed l 1 ( OWL ) Regularization Mário A. T. Figueiredo Instituto de Telecomunicações and Instituto Superior Técnico, Universidade de
More informationToeplitz Compressed Sensing Matrices with. Applications to Sparse Channel Estimation
Toeplitz Compressed Sensing Matrices with 1 Applications to Sparse Channel Estimation Jarvis Haupt, Waheed U. Bajwa, Gil Raz, and Robert Nowak Abstract Compressed sensing CS has recently emerged as a powerful
More informationUpper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices 1
Upper Bound for Intermediate Singular Values of Random Sub-Gaussian Matrices 1 Feng Wei 2 University of Michigan July 29, 2016 1 This presentation is based a project under the supervision of M. Rudelson.
More informationNear Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing
Near Ideal Behavior of a Modified Elastic Net Algorithm in Compressed Sensing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas M.Vidyasagar@utdallas.edu www.utdallas.edu/ m.vidyasagar
More informationSequential Compressed Sensing
Sequential Compressed Sensing Dmitry M. Malioutov, Sujay R. Sanghavi, and Alan S. Willsky, Fellow, IEEE Abstract Compressed sensing allows perfect recovery of sparse signals (or signals sparse in some
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationAdaptive Compressive Imaging Using Sparse Hierarchical Learned Dictionaries
Adaptive Compressive Imaging Using Sparse Hierarchical Learned Dictionaries Jarvis Haupt University of Minnesota Department of Electrical and Computer Engineering Supported by Motivation New Agile Sensing
More informationLecture 3. Random Fourier measurements
Lecture 3. Random Fourier measurements 1 Sampling from Fourier matrices 2 Law of Large Numbers and its operator-valued versions 3 Frames. Rudelson s Selection Theorem Sampling from Fourier matrices Our
More informationGREEDY SIGNAL RECOVERY REVIEW
GREEDY SIGNAL RECOVERY REVIEW DEANNA NEEDELL, JOEL A. TROPP, ROMAN VERSHYNIN Abstract. The two major approaches to sparse recovery are L 1-minimization and greedy methods. Recently, Needell and Vershynin
More informationNew ways of dimension reduction? Cutting data sets into small pieces
New ways of dimension reduction? Cutting data sets into small pieces Roman Vershynin University of Michigan, Department of Mathematics Statistical Machine Learning Ann Arbor, June 5, 2012 Joint work with
More informationUniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit
Uniform Uncertainty Principle and signal recovery via Regularized Orthogonal Matching Pursuit arxiv:0707.4203v2 [math.na] 14 Aug 2007 Deanna Needell Department of Mathematics University of California,
More informationInterpolation via weighted l 1 -minimization
Interpolation via weighted l 1 -minimization Holger Rauhut RWTH Aachen University Lehrstuhl C für Mathematik (Analysis) Matheon Workshop Compressive Sensing and Its Applications TU Berlin, December 11,
More informationFrame Coherence and Sparse Signal Processing
Frame Coherence and Sparse Signal Processing Dustin G ixon Waheed U Bajwa Robert Calderbank Program in Applied and Computational athematics Princeton University Princeton ew Jersey 08544 Department of
More informationHigh-dimensional Statistical Models
High-dimensional Statistical Models Pradeep Ravikumar UT Austin MLSS 2014 1 Curse of Dimensionality Statistical Learning: Given n observations from p(x; θ ), where θ R p, recover signal/parameter θ. For
More informationSparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery
Sparse analysis Lecture V: From Sparse Approximation to Sparse Signal Recovery Anna C. Gilbert Department of Mathematics University of Michigan Connection between... Sparse Approximation and Compressed
More informationRandom Matrices: Invertibility, Structure, and Applications
Random Matrices: Invertibility, Structure, and Applications Roman Vershynin University of Michigan Colloquium, October 11, 2011 Roman Vershynin (University of Michigan) Random Matrices Colloquium 1 / 37
More informationObservability with Random Observations
Observability with Random Observations 1 Borhan M. Sanandaji, Michael B. Wakin, and Tyrone L. Vincent Abstract Recovery of the initial state of a high-dimensional system can require a large number of measurements.
More informationSolving Underdetermined Linear Equations and Overdetermined Quadratic Equations (using Convex Programming)
Solving Underdetermined Linear Equations and Overdetermined Quadratic Equations (using Convex Programming) Justin Romberg Georgia Tech, ECE Caltech ROM-GR Workshop June 7, 2013 Pasadena, California Linear
More informationGuaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
Forty-Fifth Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 26-28, 27 WeA3.2 Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization Benjamin
More informationNoisy Signal Recovery via Iterative Reweighted L1-Minimization
Noisy Signal Recovery via Iterative Reweighted L1-Minimization Deanna Needell UC Davis / Stanford University Asilomar SSC, November 2009 Problem Background Setup 1 Suppose x is an unknown signal in R d.
More informationHigh-dimensional Statistics
High-dimensional Statistics Pradeep Ravikumar UT Austin Outline 1. High Dimensional Data : Large p, small n 2. Sparsity 3. Group Sparsity 4. Low Rank 1 Curse of Dimensionality Statistical Learning: Given
More informationRandomized sparsification in NP-hard norms
58 Chapter 3 Randomized sparsification in NP-hard norms Massive matrices are ubiquitous in modern data processing. Classical dense matrix algorithms are poorly suited to such problems because their running
More informationLecture 22: More On Compressed Sensing
Lecture 22: More On Compressed Sensing Scribed by Eric Lee, Chengrun Yang, and Sebastian Ament Nov. 2, 207 Recap and Introduction Basis pursuit was the method of recovering the sparsest solution to an
More informationSubset selection with sparse matrices
Subset selection with sparse matrices Alberto Del Pia, University of Wisconsin-Madison Santanu S. Dey, Georgia Tech Robert Weismantel, ETH Zürich February 1, 018 Schloss Dagstuhl Subset selection for regression
More informationLimitations in Approximating RIP
Alok Puranik Mentor: Adrian Vladu Fifth Annual PRIMES Conference, 2015 Outline 1 Background The Problem Motivation Construction Certification 2 Planted model Planting eigenvalues Analysis Distinguishing
More informationDeterministic constructions of compressed sensing matrices
Journal of Complexity 23 (2007) 918 925 www.elsevier.com/locate/jco Deterministic constructions of compressed sensing matrices Ronald A. DeVore Department of Mathematics, University of South Carolina,
More informationMultivariate Statistics Random Projections and Johnson-Lindenstrauss Lemma
Multivariate Statistics Random Projections and Johnson-Lindenstrauss Lemma Suppose again we have n sample points x,..., x n R p. The data-point x i R p can be thought of as the i-th row X i of an n p-dimensional
More informationTractable Upper Bounds on the Restricted Isometry Constant
Tractable Upper Bounds on the Restricted Isometry Constant Alex d Aspremont, Francis Bach, Laurent El Ghaoui Princeton University, École Normale Supérieure, U.C. Berkeley. Support from NSF, DHS and Google.
More informationTHEORY OF COMPRESSIVE SENSING VIA l 1 -MINIMIZATION: A NON-RIP ANALYSIS AND EXTENSIONS
THEORY OF COMPRESSIVE SENSING VIA l 1 -MINIMIZATION: A NON-RIP ANALYSIS AND EXTENSIONS YIN ZHANG Abstract. Compressive sensing (CS) is an emerging methodology in computational signal processing that has
More informationSparse Recovery over Graph Incidence Matrices
Sparse Recovery over Graph Incidence Matrices Mengnan Zhao, M. Devrim Kaba, René Vidal, Daniel P. Robinson, and Enrique Mallada Abstract Classical results in sparse recovery guarantee the exact reconstruction
More informationSensing systems limited by constraints: physical size, time, cost, energy
Rebecca Willett Sensing systems limited by constraints: physical size, time, cost, energy Reduce the number of measurements needed for reconstruction Higher accuracy data subject to constraints Original
More informationCompressed Sensing and Neural Networks
and Jan Vybíral (Charles University & Czech Technical University Prague, Czech Republic) NOMAD Summer Berlin, September 25-29, 2017 1 / 31 Outline Lasso & Introduction Notation Training the network Applications
More informationCoSaMP. Iterative signal recovery from incomplete and inaccurate samples. Joel A. Tropp
CoSaMP Iterative signal recovery from incomplete and inaccurate samples Joel A. Tropp Applied & Computational Mathematics California Institute of Technology jtropp@acm.caltech.edu Joint with D. Needell
More informationECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis Lecture 3: Sparse signal recovery: A RIPless analysis of l 1 minimization Yuejie Chi The Ohio State University Page 1 Outline
More informationOptimization-based sparse recovery: Compressed sensing vs. super-resolution
Optimization-based sparse recovery: Compressed sensing vs. super-resolution Carlos Fernandez-Granda, Google Computational Photography and Intelligent Cameras, IPAM 2/5/2014 This work was supported by a
More informationSparse Parameter Recovery from Aggregated Data : Supplement
Avradeep Bhowmik Joydeep Ghosh The University of Texas at Austin, TX, USA Oluwasanmi Koyejo Stanford University, CA & University of Illinois at Urbana Champaign, IL, USA AVRADEEP.1@UTEXAS.EDU GHOSH@ECE.UTEXAS.EDU
More informationSparse recovery for spherical harmonic expansions
Rachel Ward 1 1 Courant Institute, New York University Workshop Sparsity and Cosmology, Nice May 31, 2011 Cosmic Microwave Background Radiation (CMB) map Temperature is measured as T (θ, ϕ) = k k=0 l=
More informationExplicit Constructions for Compressed Sensing of Sparse Signals
Explicit Constructions for Compressed Sensing of Sparse Signals Piotr Indyk MIT July 12, 2007 1 Introduction Over the recent years, a new approach for obtaining a succinct approximate representation of
More informationLinear Sketches A Useful Tool in Streaming and Compressive Sensing
Linear Sketches A Useful Tool in Streaming and Compressive Sensing Qin Zhang 1-1 Linear sketch Random linear projection M : R n R k that preserves properties of any v R n with high prob. where k n. M =
More informationConcentration Inequalities for Random Matrices
Concentration Inequalities for Random Matrices M. Ledoux Institut de Mathématiques de Toulouse, France exponential tail inequalities classical theme in probability and statistics quantify the asymptotic
More informationTractable performance bounds for compressed sensing.
Tractable performance bounds for compressed sensing. Alex d Aspremont, Francis Bach, Laurent El Ghaoui Princeton University, École Normale Supérieure/INRIA, U.C. Berkeley. Support from NSF, DHS and Google.
More information19.1 Problem setup: Sparse linear regression
ECE598: Information-theoretic methods in high-dimensional statistics Spring 2016 Lecture 19: Minimax rates for sparse linear regression Lecturer: Yihong Wu Scribe: Subhadeep Paul, April 13/14, 2016 In
More informationSuprema of Chaos Processes and the Restricted Isometry Property
Suprema of Chaos Processes and the Restricted Isometry Property Felix Krahmer, Shahar Mendelson, and Holger Rauhut June 30, 2012 Abstract We present a new bound for suprema of a special type of chaos processes
More informationof Orthogonal Matching Pursuit
A Sharp Restricted Isometry Constant Bound of Orthogonal Matching Pursuit Qun Mo arxiv:50.0708v [cs.it] 8 Jan 205 Abstract We shall show that if the restricted isometry constant (RIC) δ s+ (A) of the measurement
More informationOptimisation Combinatoire et Convexe.
Optimisation Combinatoire et Convexe. Low complexity models, l 1 penalties. A. d Aspremont. M1 ENS. 1/36 Today Sparsity, low complexity models. l 1 -recovery results: three approaches. Extensions: matrix
More informationOn the Observability of Linear Systems from Random, Compressive Measurements
On the Observability of Linear Systems from Random, Compressive Measurements Michael B Wakin, Borhan M Sanandaji, and Tyrone L Vincent Abstract Recovering or estimating the initial state of a highdimensional
More informationGreedy Signal Recovery and Uniform Uncertainty Principles
Greedy Signal Recovery and Uniform Uncertainty Principles SPIE - IE 2008 Deanna Needell Joint work with Roman Vershynin UC Davis, January 2008 Greedy Signal Recovery and Uniform Uncertainty Principles
More informationCompressed Sensing and Affine Rank Minimization Under Restricted Isometry
IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 61, NO. 13, JULY 1, 2013 3279 Compressed Sensing Affine Rank Minimization Under Restricted Isometry T. Tony Cai Anru Zhang Abstract This paper establishes new
More informationRandomness-in-Structured Ensembles for Compressed Sensing of Images
Randomness-in-Structured Ensembles for Compressed Sensing of Images Abdolreza Abdolhosseini Moghadam Dep. of Electrical and Computer Engineering Michigan State University Email: abdolhos@msu.edu Hayder
More informationsample lectures: Compressed Sensing, Random Sampling I & II
P. Jung, CommIT, TU-Berlin]: sample lectures Compressed Sensing / Random Sampling I & II 1/68 sample lectures: Compressed Sensing, Random Sampling I & II Peter Jung Communications and Information Theory
More informationLEARNING DATA TRIAGE: LINEAR DECODING WORKS FOR COMPRESSIVE MRI. Yen-Huan Li and Volkan Cevher
LARNING DATA TRIAG: LINAR DCODING WORKS FOR COMPRSSIV MRI Yen-Huan Li and Volkan Cevher Laboratory for Information Inference Systems École Polytechnique Fédérale de Lausanne ABSTRACT The standard approach
More information