Signal Reconstruction in Linear Mixing Systems with Different Error Metrics
|
|
|
- Susanna Dixon
- 5 years ago
- Views:
Transcription
1 Signal Reconstruction in Linear Mixing Systems with Different Error Metrics Jin Tan and Dror Baron North Carolina State University Feb. 14, 2013 Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
2 Motivation Motivation Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
3 Motivation Multiuser communication (CDMA) Multiuser communication (CDMA) Users transmit according to columns of matrix Φ Users transmit according to columns of matrix Φ most most users quiet; few users active active > sparse sparse we we do don t notknow knowwhich whichusers users active active under-determined system under-determined system Receiver sees sees noisy noisy version y = Φx + z Cellphones Cellphones estimate estimate x from from y and and Φ Φ Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
4 Motivation Pervasive in Science and Engineering Pervasive in Science and Engineering Medical imaging Medical (tomography) imaging (tomography) Multiuser communication Multiuser communication (CDMA) (CDMA) Financial prediction Financial prediction Electromagnetic scattering Electromagnetic scattering Seismic imaging (oil industry) Seismic imaging (oil industry) Compressed sensing Compressed Sensing (underdetermined systems) Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
5 Problem Model Problem Model Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
6 Problem Model Mixing System x w y = channel M N;M <N M 1 M 1 Measurements y depend linearly on unknown input x Always contains errors Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
7 Problem Model Estimation x w y bx = channel estimation M N;M <N M 1 M 1 Goal: estimate x from y and Φ N 1 N 1 Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
8 Scalar estim Error Metric Problem Model N 1 N 1 ; f YjX Rela BP x D(bx; x) bx Error metric: NX Bayes rule: General form: D(bx; x) = d(bx i ; x i ) Additive error i=1 metric: Define D(bx Absolute error: D( x, d(bx i x) ; x i ) = N jbx i i=1 x d( x i j i, x i ) bx Want to minimize expected error: o Squared error: d(bx i ; x i ) = (bx i x i ) E[D( x, x)] 2 What if a non-standard metric d(bx i ; x i ) is Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
9 Scalar estim What Next? Problem Model N 1 N 1 ; f YjX Rela BP x D(bx; x) bx Error metric: NX General form: D(bx; x) = d(bx i ; x i ) i=1 Need correct error metric Absolute error: d(bx i ; x i ) = jbx i x i j Bayes rule: Define D(bx Squared error: Need d(bx i ; xposterior i ) = (bx i xinformation i ) 2 bx o What if a non-standard metric d(bx i ; x i ) is Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
10 Proposed Algorithm Metric-optimal Algorithm Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
11 Posterior Information Proposed Algorithm Background Relaxed Decoupling belief Principle propagation [Guo & Verdú, (BP) 2005; process Guo decouples & Wang, 2007] linear mixing Relaxed channels Belief Propagation to parallel scalar (BP) [Rangan, Gaussian 2010] channels. v 1 x 1 11 x 1 q 1 x 2 22 w 1 w 2 Relax BP x 2 v 2 v 3 q 2 x 3 x N 13 2N M2 w M x 3 x N v N q 3 q N w = x Linear mixing channels q i = x i + v i ; v i» N (0; ¹) Scalar Gaussian channels Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
12 Proposed Algorithm Metric-optimal Algorithm d(x i, x i ) f(x i q i ) argmin E[d(x i, x i ) q i ] x i Scalar posterior from relaxed BP User-defined additive error metric Combining posterior and error metric expected error Pointwise minimization of expected error Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
13 Properties Properties Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
14 Properties E[d(x i, x i ) q i ] x ooo,i x i Claim 1 The proposed algorithm is asymptotically optimal as the signal dimension N has the lowest possible error Metric-optimal, achieves the best possible performance Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
15 Properties I want x i x i x i x i 3.5 How about x i x i 0.77 Claim 2 The minimum mean user-defined error (MMUE) is given by ( ) MMUE(f X, µ) = D( x opt, x)f (x q)dx dq R(Q) R(X) Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
16 Numerical Results Numerical Results Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
17 Numerical Results Comparison with Other Algorithms (N = 10, 000) D( x, x) = N i=1 x i x i 0.5 Relaxed BP CoSaMP Metric optimal Error Number of measurements M Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
18 Numerical Results Comparison with Other Algorithms (N = 10, 000) D( x, x) = N i=1 x i x i Relaxed BP CoSaMP Metric optimal Error Number of measurements M Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
19 Numerical Results Comparison with Other Algorithms (N = 10, 000) D( x, x) = N i=1 x i x i Relaxed BP CoSaMP Metric optimal Error Number of measurements M Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
20 Numerical Results Comparison with Theoretical Limits (N = 10, 000) Absolute error Support error Minimum mean absolute error estimator Theoretical limit Metric optimal Number of measurements M Minimum mean support error estimator Theoretical limit Metric optimal Number of measurements M Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
21 The l norm error The l Norm Error Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
22 Definition of l norm The l norm error x x x x max x i x i i x x = max i {1,2,...,N} x i x i Want to use metric-optimal algo, but l not additive... Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
23 Definition of l norm The l norm error x x x x max x i x i i x x = max i {1,2,...,N} x i x i Want to use metric-optimal algo, but l not additive... Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
24 The l norm error Wiener Filter Gaussian signal, Gaussian noise (q = x G + v) Wiener filter optimal for l error [Sherman,1958] x Wiener = const q Sparse Gaussian signal, Gaussian noise (q = x SG + v) Wiener filter asymptotically optimal What if N is finite? Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
25 The l norm error Wiener Filter Gaussian signal, Gaussian noise (q = x G + v) Wiener filter optimal for l error [Sherman,1958] x Wiener = const q Sparse Gaussian signal, Gaussian noise (q = x SG + v) Wiener filter asymptotically optimal What if N is finite? Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
26 The l norm error Wiener Filter Gaussian signal, Gaussian noise (q = x G + v) Wiener filter optimal for l error [Sherman,1958] x Wiener = const q Sparse Gaussian signal, Gaussian noise (q = x SG + v) Wiener filter asymptotically optimal What if N is finite? Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
27 The l norm error l p Approximation to l x x = max i {1,2,...,N} x i x i = lim p x i x i p D( x, x) = N x i x i p i=1 d(x i, x i ) f(x i q i ) argmin E[d(x i, x i ) q i ] x i Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
28 The l norm error Numerical Results (M/N = 0.3) D( x, x) = N i=1 x i x i p L error Signal dimension N x 5 x 10 x 15 Wiener filter Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
29 Summary Summary Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
30 Summary d(x i, x i ) f(x i q i ) argmin E[d(x i, x i ) q i ] x i User-defined additive error metric Scalar Gaussian channel, scalar estimation Wiener filter is only optimal for N Heuristic d(x i, x i ) = x i x i p suitable for finite N Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
31 Summary Thank you! Jin Tan and Dror Baron (NCSU) Reconstruction with Different Error Metrics Feb. 14, / 28
Single-letter Characterization of Signal Estimation from Linear Measurements
Single-letter Characterization of Signal Estimation from Linear Measurements Dongning Guo Dror Baron Shlomo Shamai The work has been supported by the European Commission in the framework of the FP7 Network
Wiener Filters in Gaussian Mixture Signal Estimation with l -Norm Error
Wiener Filters in Gaussian Mixture Signal Estimation with l -Norm Error Jin Tan, Student Member, IEEE, Dror Baron, Senior Member, IEEE, and Liyi Dai, Fellow, IEEE Abstract Consider the estimation of a
Mismatched Estimation in Large Linear Systems
Mismatched Estimation in Large Linear Systems Yanting Ma, Dror Baron, Ahmad Beirami Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC 7695, USA Department
Performance Regions in Compressed Sensing from Noisy Measurements
Performance egions in Compressed Sensing from Noisy Measurements Junan Zhu and Dror Baron Department of Electrical and Computer Engineering North Carolina State University; aleigh, NC 27695, USA Email:
Mismatched Estimation in Large Linear Systems
Mismatched Estimation in Large Linear Systems Yanting Ma, Dror Baron, and Ahmad Beirami North Carolina State University Massachusetts Institute of Technology & Duke University Supported by NSF & ARO Motivation
Sparse Superposition Codes for the Gaussian Channel
Sparse Superposition Codes for the Gaussian Channel Florent Krzakala (LPS, Ecole Normale Supérieure, France) J. Barbier (ENS) arxiv:1403.8024 presented at ISIT 14 Long version in preparation Communication
5. Density evolution. Density evolution 5-1
5. Density evolution Density evolution 5-1 Probabilistic analysis of message passing algorithms variable nodes factor nodes x1 a x i x2 a(x i ; x j ; x k ) x3 b x4 consider factor graph model G = (V ;
Interactions of Information Theory and Estimation in Single- and Multi-user Communications
Interactions of Information Theory and Estimation in Single- and Multi-user Communications Dongning Guo Department of Electrical Engineering Princeton University March 8, 2004 p 1 Dongning Guo Communications
Introduction to Compressed Sensing
Introduction to Compressed Sensing Alejandro Parada, Gonzalo Arce University of Delaware August 25, 2016 Motivation: Classical Sampling 1 Motivation: Classical Sampling Issues Some applications Radar Spectral
Approximate Message Passing for Bilinear Models
Approximate Message Passing for Bilinear Models Volkan Cevher Laboratory for Informa4on and Inference Systems LIONS / EPFL h,p://lions.epfl.ch & Idiap Research Ins=tute joint work with Mitra Fatemi @Idiap
On convergence of Approximate Message Passing
On convergence of Approximate Message Passing Francesco Caltagirone (1), Florent Krzakala (2) and Lenka Zdeborova (1) (1) Institut de Physique Théorique, CEA Saclay (2) LPS, Ecole Normale Supérieure, Paris
Multipath Matching Pursuit
Multipath Matching Pursuit Submitted to IEEE trans. on Information theory Authors: S. Kwon, J. Wang, and B. Shim Presenter: Hwanchol Jang Multipath is investigated rather than a single path for a greedy
Independent Component Analysis. Contents
Contents Preface xvii 1 Introduction 1 1.1 Linear representation of multivariate data 1 1.1.1 The general statistical setting 1 1.1.2 Dimension reduction methods 2 1.1.3 Independence as a guiding principle
Image Gradients and Gradient Filtering Computer Vision
Image Gradients and Gradient Filtering 16-385 Computer Vision What is an image edge? Recall that an image is a 2D function f(x) edge edge How would you detect an edge? What kinds of filter would you use?
Decoupling of CDMA Multiuser Detection via the Replica Method
Decoupling of CDMA Multiuser Detection via the Replica Method Dongning Guo and Sergio Verdú Dept. of Electrical Engineering Princeton University Princeton, NJ 08544, USA email: {dguo,verdu}@princeton.edu
Information Dimension
Information Dimension Mina Karzand Massachusetts Institute of Technology November 16, 2011 1 / 26 2 / 26 Let X would be a real-valued random variable. For m N, the m point uniform quantized version of
Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
ANALYSIS OF MAP ESTIMATION VIA THE REPLICA METHOD AND APPLICATIONS TO COMPRESSED SENSING Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing Sundeep Rangan,
Risk and Noise Estimation in High Dimensional Statistics via State Evolution
Risk and Noise Estimation in High Dimensional Statistics via State Evolution Mohsen Bayati Stanford University Joint work with Jose Bento, Murat Erdogdu, Marc Lelarge, and Andrea Montanari Statistical
On the Optimum Asymptotic Multiuser Efficiency of Randomly Spread CDMA
On the Optimum Asymptotic Multiuser Efficiency of Randomly Spread CDMA Ralf R. Müller Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) Lehrstuhl für Digitale Übertragung 13 December 2014 1. Introduction
Performance Trade-Offs in Multi-Processor Approximate Message Passing
Performance Trade-Offs in Multi-Processor Approximate Message Passing Junan Zhu, Ahmad Beirami, and Dror Baron Department of Electrical and Computer Engineering, North Carolina State University, Email:
Implicit sampling for particle filters. Alexandre Chorin, Mathias Morzfeld, Xuemin Tu, Ethan Atkins
0/20 Implicit sampling for particle filters Alexandre Chorin, Mathias Morzfeld, Xuemin Tu, Ethan Atkins University of California at Berkeley 2/20 Example: Try to find people in a boat in the middle of
Notions such as convergent sequence and Cauchy sequence make sense for any metric space. Convergent Sequences are Cauchy
Banach Spaces These notes provide an introduction to Banach spaces, which are complete normed vector spaces. For the purposes of these notes, all vector spaces are assumed to be over the real numbers.
Bayesian Methods for Sparse Signal Recovery
Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Jason Palmer, Zhilin Zhang and Ritwik Giri Motivation Motivation Sparse Signal Recovery
The Minimax Noise Sensitivity in Compressed Sensing
The Minimax Noise Sensitivity in Compressed Sensing Galen Reeves and avid onoho epartment of Statistics Stanford University Abstract Consider the compressed sensing problem of estimating an unknown k-sparse
Expectation Propagation Algorithm
Expectation Propagation Algorithm 1 Shuang Wang School of Electrical and Computer Engineering University of Oklahoma, Tulsa, OK, 74135 Email: {shuangwang}@ou.edu This note contains three parts. First,
Statistical Mechanics of MAP Estimation: General Replica Ansatz
1 Statistical Mechanics of MAP Estimation: General Replica Ansatz Ali Bereyhi, Ralf R. Müller, and Hermann Schulz-Baldes arxiv:1612.01980v2 [cs.it 23 Oct 2017 Abstract The large-system performance of maximum-a-posterior
Inferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines. Eric W. Tramel. itwist 2016 Aalborg, DK 24 August 2016
Inferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines Eric W. Tramel itwist 2016 Aalborg, DK 24 August 2016 Andre MANOEL, Francesco CALTAGIRONE, Marylou GABRIE, Florent
Recent Developments in Compressed Sensing
Recent Developments in Compressed Sensing M. Vidyasagar Distinguished Professor, IIT Hyderabad m.vidyasagar@iith.ac.in, www.iith.ac.in/ m vidyasagar/ ISL Seminar, Stanford University, 19 April 2018 Outline
Particle Filtered Modified-CS (PaFiMoCS) for tracking signal sequences
Particle Filtered Modified-CS (PaFiMoCS) for tracking signal sequences Samarjit Das and Namrata Vaswani Department of Electrical and Computer Engineering Iowa State University http://www.ece.iastate.edu/
Dimension Reduction. David M. Blei. April 23, 2012
Dimension Reduction David M. Blei April 23, 2012 1 Basic idea Goal: Compute a reduced representation of data from p -dimensional to q-dimensional, where q < p. x 1,...,x p z 1,...,z q (1) We want to do
Vector Approximate Message Passing. Phil Schniter
Vector Approximate Message Passing Phil Schniter Collaborators: Sundeep Rangan (NYU), Alyson Fletcher (UCLA) Supported in part by NSF grant CCF-1527162. itwist @ Aalborg University Aug 24, 2016 Standard
Bayesian Paradigm. Maximum A Posteriori Estimation
Bayesian Paradigm Maximum A Posteriori Estimation Simple acquisition model noise + degradation Constraint minimization or Equivalent formulation Constraint minimization Lagrangian (unconstraint minimization)
Phil Schniter. Supported in part by NSF grants IIP , CCF , and CCF
AMP-inspired Deep Networks, with Comms Applications Phil Schniter Collaborators: Sundeep Rangan (NYU), Alyson Fletcher (UCLA), Mark Borgerding (OSU) Supported in part by NSF grants IIP-1539960, CCF-1527162,
y Xw 2 2 y Xw λ w 2 2
CS 189 Introduction to Machine Learning Spring 2018 Note 4 1 MLE and MAP for Regression (Part I) So far, we ve explored two approaches of the regression framework, Ordinary Least Squares and Ridge Regression:
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems
1 Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems Alyson K. Fletcher, Mojtaba Sahraee-Ardakan, Philip Schniter, and Sundeep Rangan Abstract arxiv:1706.06054v1 cs.it
Introduction to Graphical Models. Srikumar Ramalingam School of Computing University of Utah
Introduction to Graphical Models Srikumar Ramalingam School of Computing University of Utah Reference Christopher M. Bishop, Pattern Recognition and Machine Learning, Jonathan S. Yedidia, William T. Freeman,
Introduction to Gaussian Processes
Introduction to Gaussian Processes 1 Objectives to express prior knowledge/beliefs about model outputs using Gaussian process (GP) to sample functions from the probability measure defined by GP to build
Sparse Solutions of an Undetermined Linear System
1 Sparse Solutions of an Undetermined Linear System Maddullah Almerdasy New York University Tandon School of Engineering arxiv:1702.07096v1 [math.oc] 23 Feb 2017 Abstract This work proposes a research
Generalized Orthogonal Matching Pursuit- A Review and Some
Generalized Orthogonal Matching Pursuit- A Review and Some New Results Department of Electronics and Electrical Communication Engineering Indian Institute of Technology, Kharagpur, INDIA Table of Contents
A Fresh Look at the Bayes Theorem from Information Theory
A Fresh Look at the Bayes Theorem from Information Theory Tan Bui-Thanh Computational Engineering and Optimization (CEO) Group Department of Aerospace Engineering and Engineering Mechanics Institute for
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis
ECE 8201: Low-dimensional Signal Models for High-dimensional Data Analysis Lecture 7: Matrix completion Yuejie Chi The Ohio State University Page 1 Reference Guaranteed Minimum-Rank Solutions of Linear
Motivation Sparse Signal Recovery is an interesting area with many potential applications. Methods developed for solving sparse signal recovery proble
Bayesian Methods for Sparse Signal Recovery Bhaskar D Rao 1 University of California, San Diego 1 Thanks to David Wipf, Zhilin Zhang and Ritwik Giri Motivation Sparse Signal Recovery is an interesting
Asymptotic Analysis of MAP Estimation via the Replica Method and Compressed Sensing
Asymptotic Analysis of MAP Estimation via the Replica Method and Compressed Sensing Sundeep Rangan Qualcomm Technologies Bedminster, NJ srangan@qualcomm.com Alyson K. Fletcher University of California,
Fast Reconstruction Algorithms for Deterministic Sensing Matrices and Applications
Fast Reconstruction Algorithms for Deterministic Sensing Matrices and Applications Program in Applied and Computational Mathematics Princeton University NJ 08544, USA. Introduction What is Compressive
Least Sparsity of p-norm based Optimization Problems with p > 1
Least Sparsity of p-norm based Optimization Problems with p > Jinglai Shen and Seyedahmad Mousavi Original version: July, 07; Revision: February, 08 Abstract Motivated by l p -optimization arising from
Optimality of Large MIMO Detection via Approximate Message Passing
ptimality of Large MIM Detection via Approximate Message Passing Charles Jeon, Ramina Ghods, Arian Maleki, and Christoph Studer arxiv:5.695v [cs.it] ct 5 Abstract ptimal data detection in multiple-input
Introduction to Graphical Models. Srikumar Ramalingam School of Computing University of Utah
Introduction to Graphical Models Srikumar Ramalingam School of Computing University of Utah Reference Christopher M. Bishop, Pattern Recognition and Machine Learning, Jonathan S. Yedidia, William T. Freeman,
Passing and Interference Coordination
Generalized Approximate Message Passing and Interference Coordination Sundeep Rangan, Polytechnic Institute of NYU Joint work with Alyson Fletcher (Berkeley), Vivek Goyal (MIT), Ulugbek Kamilov (EPFL/MIT),
The Kalman Filter ImPr Talk
The Kalman Filter ImPr Talk Ged Ridgway Centre for Medical Image Computing November, 2006 Outline What is the Kalman Filter? State Space Models Kalman Filter Overview Bayesian Updating of Estimates Kalman
Statistical Mechanics of MAP Estimation: General Replica Ansatz
1 Statistical Mechanics of MAP Estimation: General Replica Ansatz Ali Bereyhi, Ralf R. Müller, and Hermann Schulz-Baldes arxiv:1612.01980v1 [cs.it 6 Dec 2016 Abstract The large-system performance of maximum-a-posterior
Multi User Detection I
January 12, 2005 Outline Overview Multiple Access Communication Motivation: What is MU Detection? Overview of DS/CDMA systems Concept and Codes used in CDMA CDMA Channels Models Synchronous and Asynchronous
MMSE Dimension. snr. 1 We use the following asymptotic notation: f(x) = O (g(x)) if and only
MMSE Dimension Yihong Wu Department of Electrical Engineering Princeton University Princeton, NJ 08544, USA Email: yihongwu@princeton.edu Sergio Verdú Department of Electrical Engineering Princeton University
MMSE Denoising of 2-D Signals Using Consistent Cycle Spinning Algorithm
Denoising of 2-D Signals Using Consistent Cycle Spinning Algorithm Bodduluri Asha, B. Leela kumari Abstract: It is well known that in a real world signals do not exist without noise, which may be negligible
Noisy Signal Recovery via Iterative Reweighted L1-Minimization
Noisy Signal Recovery via Iterative Reweighted L1-Minimization Deanna Needell UC Davis / Stanford University Asilomar SSC, November 2009 Problem Background Setup 1 Suppose x is an unknown signal in R d.
Universal MAP Estimation in Compressed Sensing
Universal MAP Estimation in Compressed Sensing Dror Baron Electrical and Computer Engineering North Carolina State University Raleigh, NC 27695 Email: barondror@ncsuedu Marco F Duarte Electrical and Computer
Applied Analysis (APPM 5440): Final exam 1:30pm 4:00pm, Dec. 14, Closed books.
Applied Analysis APPM 44: Final exam 1:3pm 4:pm, Dec. 14, 29. Closed books. Problem 1: 2p Set I = [, 1]. Prove that there is a continuous function u on I such that 1 ux 1 x sin ut 2 dt = cosx, x I. Define
10. Multi-objective least squares
L Vandenberghe ECE133A (Winter 2018) 10 Multi-objective least squares multi-objective least squares regularized data fitting control estimation and inversion 10-1 Multi-objective least squares we have
An Overview of Multi-Processor Approximate Message Passing
An Overview of Multi-Processor Approximate Message Passing Junan Zhu, Ryan Pilgrim, and Dror Baron JPMorgan Chase & Co., New York, NY 10001, Email: jzhu9@ncsu.edu Department of Electrical and Computer
Wavelets in Scattering Calculations
Wavelets in Scattering Calculations W. P., Brian M. Kessler, Gerald L. Payne polyzou@uiowa.edu The University of Iowa Wavelets in Scattering Calculations p.1/43 What are Wavelets? Orthonormal basis functions.
Cambridge University Press The Mathematics of Signal Processing Steven B. Damelin and Willard Miller Excerpt More information
Introduction Consider a linear system y = Φx where Φ can be taken as an m n matrix acting on Euclidean space or more generally, a linear operator on a Hilbert space. We call the vector x a signal or input,
Math 328 Course Notes
Math 328 Course Notes Ian Robertson March 3, 2006 3 Properties of C[0, 1]: Sup-norm and Completeness In this chapter we are going to examine the vector space of all continuous functions defined on the
Bayes Classifiers. CAP5610 Machine Learning Instructor: Guo-Jun QI
Bayes Classifiers CAP5610 Machine Learning Instructor: Guo-Jun QI Recap: Joint distributions Joint distribution over Input vector X = (X 1, X 2 ) X 1 =B or B (drinking beer or not) X 2 = H or H (headache
Real Analysis Problems
Real Analysis Problems Cristian E. Gutiérrez September 14, 29 1 1 CONTINUITY 1 Continuity Problem 1.1 Let r n be the sequence of rational numbers and Prove that f(x) = 1. f is continuous on the irrationals.
Inverse problem and optimization
Inverse problem and optimization Laurent Condat, Nelly Pustelnik CNRS, Gipsa-lab CNRS, Laboratoire de Physique de l ENS de Lyon Decembre, 15th 2016 Inverse problem and optimization 2/36 Plan 1. Examples
Approximate Message Passing Algorithms
November 4, 2017 Outline AMP (Donoho et al., 2009, 2010a) Motivations Derivations from a message-passing perspective Limitations Extensions Generalized Approximate Message Passing (GAMP) (Rangan, 2011)
Optimal Data Detection in Large MIMO
1 ptimal Data Detection in Large MIM Charles Jeon, Ramina Ghods, Arian Maleki, and Christoph Studer Abstract arxiv:1811.01917v1 [cs.it] 5 Nov 018 Large multiple-input multiple-output (MIM) appears in massive
Gaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y)
Gaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y) E{(X-Y) 2 } D
Probabilistic Graphical Models
Probabilistic Graphical Models Introduction. Basic Probability and Bayes Volkan Cevher, Matthias Seeger Ecole Polytechnique Fédérale de Lausanne 26/9/2011 (EPFL) Graphical Models 26/9/2011 1 / 28 Outline
CS281 Section 4: Factor Analysis and PCA
CS81 Section 4: Factor Analysis and PCA Scott Linderman At this point we have seen a variety of machine learning models, with a particular emphasis on models for supervised learning. In particular, we
9 Forward-backward algorithm, sum-product on factor graphs
Massachusetts Institute of Technology Department of Electrical Engineering and Computer Science 6.438 Algorithms For Inference Fall 2014 9 Forward-backward algorithm, sum-product on factor graphs The previous
Part IV Compressed Sensing
Aisenstadt Chair Course CRM September 2009 Part IV Compressed Sensing Stéphane Mallat Centre de Mathématiques Appliquées Ecole Polytechnique Conclusion to Super-Resolution Sparse super-resolution is sometime
Multiuser Capacity in Block Fading Channel
Multiuser Capacity in Block Fading Channel April 2003 1 Introduction and Model We use a block-fading model, with coherence interval T where M independent users simultaneously transmit to a single receiver
SEISMIC WAVE PROPAGATION. Lecture 2: Fourier Analysis
SEISMIC WAVE PROPAGATION Lecture 2: Fourier Analysis Fourier Series & Fourier Transforms Fourier Series Review of trigonometric identities Analysing the square wave Fourier Transform Transforms of some
Approximate Message Passing
Approximate Message Passing Mohammad Emtiyaz Khan CS, UBC February 8, 2012 Abstract In this note, I summarize Sections 5.1 and 5.2 of Arian Maleki s PhD thesis. 1 Notation We denote scalars by small letters
Cell throughput analysis of the Proportional Fair scheduler in the single cell environment
Cell throughput analysis of the Proportional Fair scheduler in the single cell environment Jin-Ghoo Choi and Seawoong Bahk IEEE Trans on Vehicular Tech, Mar 2007 *** Presented by: Anh H. Nguyen February
Sum and Product Rules
Sum and Product Rules Exercise. Consider tossing a coin five times. What is the probability of getting the same result on the first two tosses or the last two tosses? Solution. Let E be the event that
Multiuser Detection of Sparsely Spread CDMA
SUBMITTED TO IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS Multiuser Detection of Sparsely Spread CDMA Dongning Guo, Member, IEEE, and Chih-Chun Wang, Member, IEEE Abstract Code-division multiple access
These outputs can be written in a more convenient form: with y(i) = Hc m (i) n(i) y(i) = (y(i); ; y K (i)) T ; c m (i) = (c m (i); ; c m K(i)) T and n
Binary Codes for synchronous DS-CDMA Stefan Bruck, Ulrich Sorger Institute for Network- and Signal Theory Darmstadt University of Technology Merckstr. 25, 6428 Darmstadt, Germany Tel.: 49 65 629, Fax:
Measurements vs. Bits: Compressed Sensing meets Information Theory
Measurements vs. Bits: Compressed Sensing meets Information Theory Shriram Sarvotham, Dror Baron, and Richard G. Baraniuk Department of Electrical and Computer Engineering Rice University, Houston, TX
EVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER
EVALUATING SYMMETRIC INFORMATION GAP BETWEEN DYNAMICAL SYSTEMS USING PARTICLE FILTER Zhen Zhen 1, Jun Young Lee 2, and Abdus Saboor 3 1 Mingde College, Guizhou University, China zhenz2000@21cn.com 2 Department
Solutions to homework assignment #7 Math 119B UC Davis, Spring for 1 r 4. Furthermore, the derivative of the logistic map is. L r(x) = r(1 2x).
Solutions to homework assignment #7 Math 9B UC Davis, Spring 0. A fixed point x of an interval map T is called superstable if T (x ) = 0. Find the value of 0 < r 4 for which the logistic map L r has a
arxiv: v2 [cs.it] 12 Jul 2017
![arxiv: v2 [cs.it] 12 Jul 2017](/thumbs/74/70684556.jpg "arxiv: v2 [cs.it] 12 Jul 2017")
ABSTRACT ZHU, JUNAN. Statistical Physics and Information Theory Perspectives on Linear Inverse Problems. (Under the direction of Dror Baron.) arxiv:1705.05070v2 [cs.it] 12 Jul 2017 Many real-world problems
Compressive sampling, or how to get something from almost nothing (probably)
Compressive sampling p. 1/13 Compressive sampling, or how to get something from almost nothing (probably) Willard Miller miller@ima.umn.edu University of Minnesota The problem 1 A signal is a real n-tuple
Parameter Estimation
1 / 44 Parameter Estimation Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay October 25, 2012 Motivation System Model used to Derive
Large System Analysis of Projection Based Algorithms for the MIMO Broadcast Channel
Large System Analysis of Projection Based Algorithms for the MIMO Broadcast Channel Christian Guthy and Wolfgang Utschick Associate Institute for Signal Processing, Technische Universität München Joint
Machine Learning for Signal Processing Sparse and Overcomplete Representations. Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013
Machine Learning for Signal Processing Sparse and Overcomplete Representations Bhiksha Raj (slides from Sourish Chaudhuri) Oct 22, 2013 1 Key Topics in this Lecture Basics Component-based representations
CSCI5654 (Linear Programming, Fall 2013) Lectures Lectures 10,11 Slide# 1
CSCI5654 (Linear Programming, Fall 2013) Lectures 10-12 Lectures 10,11 Slide# 1 Today s Lecture 1. Introduction to norms: L 1,L 2,L. 2. Casting absolute value and max operators. 3. Norm minimization problems.
SCRIBERS: SOROOSH SHAFIEEZADEH-ABADEH, MICHAËL DEFFERRARD
EE-731: ADVANCED TOPICS IN DATA SCIENCES LABORATORY FOR INFORMATION AND INFERENCE SYSTEMS SPRING 2016 INSTRUCTOR: VOLKAN CEVHER SCRIBERS: SOROOSH SHAFIEEZADEH-ABADEH, MICHAËL DEFFERRARD STRUCTURED SPARSITY
Linear and Nonlinear Iterative Multiuser Detection
1 Linear and Nonlinear Iterative Multiuser Detection Alex Grant and Lars Rasmussen University of South Australia October 2011 Outline 1 Introduction 2 System Model 3 Multiuser Detection 4 Interference
FAULT identification is the task of determining which of a
SUBMITTED TO IEEE TRANS. ON SIGNAL PROCESSING 1 Fault Identification via Non-parametric Belief Propagation Danny Bickson, Member, IEEE, Dror Baron, Senior Member, IEEE, Alexander Ihler, Member, IEEE, Harel
Turbo-AMP: A Graphical-Models Approach to Compressive Inference
Turbo-AMP: A Graphical-Models Approach to Compressive Inference Phil Schniter (With support from NSF CCF-1018368 and DARPA/ONR N66001-10-1-4090.) June 27, 2012 1 Outline: 1. Motivation. (a) the need for
Approximate Message Passing with Built-in Parameter Estimation for Sparse Signal Recovery
Approimate Message Passing with Built-in Parameter Estimation for Sparse Signal Recovery arxiv:1606.00901v1 [cs.it] Jun 016 Shuai Huang, Trac D. Tran Department of Electrical and Computer Engineering Johns
Stopping Condition for Greedy Block Sparse Signal Recovery
Stopping Condition for Greedy Block Sparse Signal Recovery Yu Luo, Ronggui Xie, Huarui Yin, and Weidong Wang Department of Electronics Engineering and Information Science, University of Science and Technology
AMS 212A Applied Mathematical Methods I Appendices of Lecture 06 Copyright by Hongyun Wang, UCSC. ( ) cos2
AMS 22A Applied Mathematical Methods I Appendices of Lecture 06 Copyright by Hongyun Wang UCSC Appendix A: Proof of Lemma Lemma : Let (x ) be the solution of x ( r( x)+ q( x) )sin 2 + ( a) 0 < cos2 where
Bayesian Regression Linear and Logistic Regression
When we want more than point estimates Bayesian Regression Linear and Logistic Regression Nicole Beckage Ordinary Least Squares Regression and Lasso Regression return only point estimates But what if we
On the Optimality of Treating Interference as Noise in Competitive Scenarios
On the Optimality of Treating Interference as Noise in Competitive Scenarios A. DYTSO, D. TUNINETTI, N. DEVROYE WORK PARTIALLY FUNDED BY NSF UNDER AWARD 1017436 OUTLINE CHANNEL MODEL AND PAST WORK ADVANTAGES
Math 205b Homework 2 Solutions
Math 5b Homework Solutions January 5, 5 Problem (R-S, II.) () For the R case, we just expand the right hand side and use the symmetry of the inner product: ( x y x y ) = = ((x, x) (y, y) (x, y) (y, x)
An Approximate Message Passing Framework for Side Information
An Approximate Message Passing Framework for Side Information Anna Ma 1, You (Joe) Zhou, Cynthia Rush 3, Dror Baron, and Deanna Needell 4 arxiv:1807.04839v1 [cs.it] 1 Jul 018 1 Department of Mathematics,
CS Lecture 19. Exponential Families & Expectation Propagation
CS 6347 Lecture 19 Exponential Families & Expectation Propagation Discrete State Spaces We have been focusing on the case of MRFs over discrete state spaces Probability distributions over discrete spaces