Mismatched Estimation in Large Linear Systems
|
|
- Cornelius Blair
- 5 years ago
- Views:
Transcription
1 Mismatched Estimation in Large Linear Systems Yanting Ma, Dror Baron, and Ahmad Beirami North Carolina State University Massachusetts Institute of Technology & Duke University Supported by NSF & ARO
2 Motivation True prior minimum mean square error (MMSE) True prior unknown in practical problems use postulated prior different from true prior mismatch in prior Mismatched estimation mean square error (MSE) bigger than MMSE excess mean square error (EMSE)
3 Motivation Mismatched estimation in scalar channels [Verdú 010] Many practical problems are modeled by linear systems (e.g., medical imaging, compressed sensing) Extension to large linear systems y x σ z z y A x σ z z = + = + scalar channel linear system
4 Simplification y x σ z z = + x, z i.i.d. Y X σ z Z = +
5 Mismatch in Scalar Channels [Verdú 010] Additive white Gaussian noise channel: Y = X + σ z Z R; X~P, Z~N(0,1) = + Y X σ z Z
6 Mismatch in Scalar Channels [Verdú 010] Additive white Gaussian noise channel: Y = X + σ z Z R; X~P, Z~N(0,1) = + Y X σ z Z True prior P: Conditional expectation: X P Y = E P X Y Mean square error (MSE): MSE P (σ z ) = E X P Y X minimum MSE (MMSE): MMSE(σ z ) = MSE P (σ z )
7 Mismatch in Scalar Channels [Verdú 010] Additive white Gaussian noise channel: Y = X + σ z Z R; X~P, Z~N(0,1) Y X σ z Z = + True prior P: Conditional expectation: X P Y = E P X Y Mean square error (MSE): MSE P (σ z ) = E X P Y X minimum MSE (MMSE): MMSE(σ z ) = MSE P (σ z ) Mismatched prior Q: MSE: MSE Q (σ z ) = E X Q Y X Bigger than MMSE
8 Mismatch in Scalar Channels [Verdú 010] Excess MSE of scalar channel (S-EMSE): S-EMSE(σ z ) = MSE Q (σ z ) MMSE(σ z ) D(P Q) = 1 0 S EMSE 1/γ dγ, γ = 1/σz relative entropy
9 Linear System Linear system: y = Ax + σ z z R M Input signal: x R N, x j ~P Noise: z i ~N(0,1) Measurement matrix: A R M N, E A ij = 0, Var A ij = 1 M σ z z
10 Decoupling [Tanaka 00, Guo & Verdú 005, ] Linear system: y = Ax + σ z z decoupling N, M N δ large system limit Scalar channel: y i = x i + σ P z i x i ~P, z i ~N(0,1) Fixed point equation: δ σ P σ z = MSE P σ p
11 Problem Statement
12 MMSE in Large Linear Systems Focusing on large linear systems True prior P: Conditional expectation: x P y, A = E P x y, A MSE: LMSE P σ z 1 = lim E x N N P y, A x = MMSE(σ z ) linear system
13 LMSE via Decoupling [Bernoulli-Gaussian input] LMSE P σ z (MMSE) Fixed point equation: δ σ σ z = MSE P σ with solution σ P
14 Mismatch in Large Linear Systems Mismatched prior Q: MSE: LMSE Q σ z 1 = lim E x N N Q y, A x Excess MSE of linear system: L-EMSE(σ z )=LMSE Q σ z MMSE(σ z )
15 Mismatch in Large Linear Systems LMSE Q σ z LMSE P σ z (MMSE) Goal: relate L-EMSE(σ z ) to S-EMSE(σ P )
16 Main Results
17 Main Result (derivation in paper) L-EMSE σ z = S-EMSE σ P + σp σ P + 1 δ L EMSE(σ z ) d dσ MSE Q σ dσ
18 Main Result (derivation in paper) L-EMSE σ z = S-EMSE σ P + σp σ P + 1 δ L EMSE(σ z ) Taylor expansion d dσ MSE Q σ dσ
19 Taylor Approximations First order approximation: L-EMSE σ z = δ δ α S-EMSE σ p + o(δ) Second order approximation: L EMSE σ z = δ δ α S EMSE σ p β δ α S EMSE σ p + o(δ ) Δ = σ Q σ P α: first derivative of MSE Q (σ ) at σ P β: second derivative of MSE Q (σ ) at σ P
20 Taylor Approximations First order approximation: L-EMSE σ z = δ δ α S-EMSE σ p + o(δ) Second order approximation: L EMSE σ z = δ δ α S EMSE σ p β δ α S EMSE σ p + o(δ ) Δ = σ Q σ P α: first derivative of MSE Q (σ ) at σ P β: second derivative of MSE Q (σ ) at σ P
21 Numerical Examples
22 Example 1: Bernoulli-Gaussian input f X x = θ N 0,1 + 1 θ δ 0 (x)
23 Example : Markov Source Two state Markov source with state space {0,1} For non-i.i.d. input (e.g., Markov source) decoupling principal not well-understood Alternative tool: state evolution (SE) of approximate message passing (AMP) [Bayati & Montanari 011]
24 Approximate Message Passing [Donoho et al. 009] AMP solves linear system by iterative decoupling Decoupled scalar channel characterized by state evolution (SE) t SE for AMP with E x i y i coincides with fixed point equation for information theoretic decoupling For non-i.i.d. input E x i window i better than E x i y i t SE for AMP with E x i window i not rigorous but numerically verified [Ma et al. 014] y x σ z z window = +
25 Example : Markov Source Two state Markov source with state space {0,1} W(3): E x i window i with window-size (3) δ σ W3 σ z = MSE W3 σ W3 Y = X + σ W3 Z
26 Example : Markov Source Two state Markov source with state space {0,1} W(3): E x i window i with window-size (3) δ σ W3 σ z = MSE W3 σ W3 Apply W to Y = X + σ W3 Z
27 Example : Markov Source Two state Markov source with state space {0,1} W(3): E x i window i with window-size (3) δ σ W σ z = MSE W σ W δ σ W3 σ z = MSE W3 σ W3 Y = X + σ W3 Z
28 Example : Markov Source Two state Markov source with state space {0,1} W(3): E x i window i with window-size (3) δ σ W3 σ z = MSE W3 σ W3 Y = X + σ W3 Z
29 Discussion
30 Summary Provided expression relating L-EMSE to S-EMSE Two Taylor approximations derived and accuracy verified numerically Next step How does parameter estimation error affect EMSE in linear systems?
31 Thank you!
Mismatched Estimation in Large Linear Systems
Mismatched Estimation in Large Linear Systems Yanting Ma, Dror Baron, Ahmad Beirami Department of Electrical and Computer Engineering, North Carolina State University, Raleigh, NC 7695, USA Department
More informationPerformance Trade-Offs in Multi-Processor Approximate Message Passing
Performance Trade-Offs in Multi-Processor Approximate Message Passing Junan Zhu, Ahmad Beirami, and Dror Baron Department of Electrical and Computer Engineering, North Carolina State University, Email:
More informationSparse Superposition Codes for the Gaussian Channel
Sparse Superposition Codes for the Gaussian Channel Florent Krzakala (LPS, Ecole Normale Supérieure, France) J. Barbier (ENS) arxiv:1403.8024 presented at ISIT 14 Long version in preparation Communication
More informationSingle-letter Characterization of Signal Estimation from Linear Measurements
Single-letter Characterization of Signal Estimation from Linear Measurements Dongning Guo Dror Baron Shlomo Shamai The work has been supported by the European Commission in the framework of the FP7 Network
More informationPerformance Regions in Compressed Sensing from Noisy Measurements
Performance egions in Compressed Sensing from Noisy Measurements Junan Zhu and Dror Baron Department of Electrical and Computer Engineering North Carolina State University; aleigh, NC 27695, USA Email:
More informationSignal Reconstruction in Linear Mixing Systems with Different Error Metrics
Signal Reconstruction in Linear Mixing Systems with Different Error Metrics Jin Tan and Dror Baron North Carolina State University Feb. 14, 2013 Jin Tan and Dror Baron (NCSU) Reconstruction with Different
More informationRisk and Noise Estimation in High Dimensional Statistics via State Evolution
Risk and Noise Estimation in High Dimensional Statistics via State Evolution Mohsen Bayati Stanford University Joint work with Jose Bento, Murat Erdogdu, Marc Lelarge, and Andrea Montanari Statistical
More informationmuch more on minimax (order bounds) cf. lecture by Iain Johnstone
much more on minimax (order bounds) cf. lecture by Iain Johnstone http://www-stat.stanford.edu/~imj/wald/wald1web.pdf today s lecture parametric estimation, Fisher information, Cramer-Rao lower bound:
More informationInteractions of Information Theory and Estimation in Single- and Multi-user Communications
Interactions of Information Theory and Estimation in Single- and Multi-user Communications Dongning Guo Department of Electrical Engineering Princeton University March 8, 2004 p 1 Dongning Guo Communications
More informationAn Overview of Multi-Processor Approximate Message Passing
An Overview of Multi-Processor Approximate Message Passing Junan Zhu, Ryan Pilgrim, and Dror Baron JPMorgan Chase & Co., New York, NY 10001, Email: jzhu9@ncsu.edu Department of Electrical and Computer
More informationMMSE Dimension. snr. 1 We use the following asymptotic notation: f(x) = O (g(x)) if and only
MMSE Dimension Yihong Wu Department of Electrical Engineering Princeton University Princeton, NJ 08544, USA Email: yihongwu@princeton.edu Sergio Verdú Department of Electrical Engineering Princeton University
More informationThe Minimax Noise Sensitivity in Compressed Sensing
The Minimax Noise Sensitivity in Compressed Sensing Galen Reeves and avid onoho epartment of Statistics Stanford University Abstract Consider the compressed sensing problem of estimating an unknown k-sparse
More informationMulti-Processor Approximate Message Passing with Lossy Compression
1 Multi-rocessor Approximate Message assing with Lossy Compression Junan Zhu, Student Member, IEEE, and Dror Baron, Senior Member, IEEE Abstract We consider large-scale linear inverse problems in Bayesian
More informationarxiv: v2 [cs.it] 12 Jul 2017
ABSTRACT ZHU, JUNAN. Statistical Physics and Information Theory Perspectives on Linear Inverse Problems. (Under the direction of Dror Baron.) arxiv:1705.05070v2 [cs.it] 12 Jul 2017 Many real-world problems
More informationCompressive Sensing under Matrix Uncertainties: An Approximate Message Passing Approach
Compressive Sensing under Matrix Uncertainties: An Approximate Message Passing Approach Asilomar 2011 Jason T. Parker (AFRL/RYAP) Philip Schniter (OSU) Volkan Cevher (EPFL) Problem Statement Traditional
More informationApproximate Message Passing for Bilinear Models
Approximate Message Passing for Bilinear Models Volkan Cevher Laboratory for Informa4on and Inference Systems LIONS / EPFL h,p://lions.epfl.ch & Idiap Research Ins=tute joint work with Mitra Fatemi @Idiap
More informationAn Approximate Message Passing Framework for Side Information
An Approximate Message Passing Framework for Side Information Anna Ma 1, You (Joe) Zhou, Cynthia Rush 3, Dror Baron, and Deanna Needell 4 arxiv:1807.04839v1 [cs.it] 1 Jul 018 1 Department of Mathematics,
More informationPassing and Interference Coordination
Generalized Approximate Message Passing and Interference Coordination Sundeep Rangan, Polytechnic Institute of NYU Joint work with Alyson Fletcher (Berkeley), Vivek Goyal (MIT), Ulugbek Kamilov (EPFL/MIT),
More informationInformation-theoretically Optimal Sparse PCA
Information-theoretically Optimal Sparse PCA Yash Deshpande Department of Electrical Engineering Stanford, CA. Andrea Montanari Departments of Electrical Engineering and Statistics Stanford, CA. Abstract
More informationWiener Filters in Gaussian Mixture Signal Estimation with l -Norm Error
Wiener Filters in Gaussian Mixture Signal Estimation with l -Norm Error Jin Tan, Student Member, IEEE, Dror Baron, Senior Member, IEEE, and Liyi Dai, Fellow, IEEE Abstract Consider the estimation of a
More informationRigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems
1 Rigorous Dynamics and Consistent Estimation in Arbitrarily Conditioned Linear Systems Alyson K. Fletcher, Mojtaba Sahraee-Ardakan, Philip Schniter, and Sundeep Rangan Abstract arxiv:1706.06054v1 cs.it
More informationOn convergence of Approximate Message Passing
On convergence of Approximate Message Passing Francesco Caltagirone (1), Florent Krzakala (2) and Lenka Zdeborova (1) (1) Institut de Physique Théorique, CEA Saclay (2) LPS, Ecole Normale Supérieure, Paris
More informationPhil Schniter. Supported in part by NSF grants IIP , CCF , and CCF
AMP-inspired Deep Networks, with Comms Applications Phil Schniter Collaborators: Sundeep Rangan (NYU), Alyson Fletcher (UCLA), Mark Borgerding (OSU) Supported in part by NSF grants IIP-1539960, CCF-1527162,
More informationOptimal Data Detection in Large MIMO
1 ptimal Data Detection in Large MIM Charles Jeon, Ramina Ghods, Arian Maleki, and Christoph Studer Abstract arxiv:1811.01917v1 [cs.it] 5 Nov 018 Large multiple-input multiple-output (MIM) appears in massive
More informationOn the Achievable Rates of Decentralized Equalization in Massive MU-MIMO Systems
On the Achievable Rates of Decentralized Equalization in Massive MU-MIMO Systems Charles Jeon, Kaipeng Li, Joseph R. Cavallaro, and Christoph Studer Abstract Massive multi-user (MU) multiple-input multipleoutput
More informationInferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines. Eric W. Tramel. itwist 2016 Aalborg, DK 24 August 2016
Inferring Sparsity: Compressed Sensing Using Generalized Restricted Boltzmann Machines Eric W. Tramel itwist 2016 Aalborg, DK 24 August 2016 Andre MANOEL, Francesco CALTAGIRONE, Marylou GABRIE, Florent
More informationDecoupling of CDMA Multiuser Detection via the Replica Method
Decoupling of CDMA Multiuser Detection via the Replica Method Dongning Guo and Sergio Verdú Dept. of Electrical Engineering Princeton University Princeton, NJ 08544, USA email: {dguo,verdu}@princeton.edu
More informationTurbo-AMP: A Graphical-Models Approach to Compressive Inference
Turbo-AMP: A Graphical-Models Approach to Compressive Inference Phil Schniter (With support from NSF CCF-1018368 and DARPA/ONR N66001-10-1-4090.) June 27, 2012 1 Outline: 1. Motivation. (a) the need for
More informationApproximate Message Passing Algorithms
November 4, 2017 Outline AMP (Donoho et al., 2009, 2010a) Motivations Derivations from a message-passing perspective Limitations Extensions Generalized Approximate Message Passing (GAMP) (Rangan, 2011)
More informationOptimality of Large MIMO Detection via Approximate Message Passing
ptimality of Large MIM Detection via Approximate Message Passing Charles Jeon, Ramina Ghods, Arian Maleki, and Christoph Studer arxiv:5.695v [cs.it] ct 5 Abstract ptimal data detection in multiple-input
More informationLecture 20: Quantization and Rate-Distortion
Lecture 20: Quantization and Rate-Distortion Quantization Introduction to rate-distortion theorem Dr. Yao Xie, ECE587, Information Theory, Duke University Approimating continuous signals... Dr. Yao Xie,
More informationarxiv: v2 [cs.it] 6 Sep 2016
The Mutual Information in Random Linear Estimation Jean Barbier, Mohamad Dia, Nicolas Macris and Florent Krzakala Laboratoire de Théorie des Communications, Faculté Informatique et Communications, Ecole
More informationBayesian statistics. DS GA 1002 Statistical and Mathematical Models. Carlos Fernandez-Granda
Bayesian statistics DS GA 1002 Statistical and Mathematical Models http://www.cims.nyu.edu/~cfgranda/pages/dsga1002_fall15 Carlos Fernandez-Granda Frequentist vs Bayesian statistics In frequentist statistics
More informationStatistics: Learning models from data
DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial
More informationApproximate Message Passing with Built-in Parameter Estimation for Sparse Signal Recovery
Approimate Message Passing with Built-in Parameter Estimation for Sparse Signal Recovery arxiv:1606.00901v1 [cs.it] Jun 016 Shuai Huang, Trac D. Tran Department of Electrical and Computer Engineering Johns
More informationLecture 22: Final Review
Lecture 22: Final Review Nuts and bolts Fundamental questions and limits Tools Practical algorithms Future topics Dr Yao Xie, ECE587, Information Theory, Duke University Basics Dr Yao Xie, ECE587, Information
More informationOn the Shamai-Laroia Approximation for the Information Rate of the ISI Channel
On the Shamai-Laroia Approximation for the Information Rate of the ISI Channel Yair Carmon and Shlomo Shamai (Shitz) Department of Electrical Engineering, Technion - Israel Institute of Technology 2014
More informationInformation Dimension
Information Dimension Mina Karzand Massachusetts Institute of Technology November 16, 2011 1 / 26 2 / 26 Let X would be a real-valued random variable. For m N, the m point uniform quantized version of
More informationDS-GA 1002 Lecture notes 11 Fall Bayesian statistics
DS-GA 100 Lecture notes 11 Fall 016 Bayesian statistics In the frequentist paradigm we model the data as realizations from a distribution that depends on deterministic parameters. In contrast, in Bayesian
More informationAsymptotic Analysis of MAP Estimation via the Replica Method and Compressed Sensing
Asymptotic Analysis of MAP Estimation via the Replica Method and Compressed Sensing Sundeep Rangan Qualcomm Technologies Bedminster, NJ srangan@qualcomm.com Alyson K. Fletcher University of California,
More informationPerformance Limits for Noisy Multi-Measurement Vector Problems
Performance Limits for Noisy Multi-Measurement Vector Problems Junan Zhu, Student Member, IEEE, Dror Baron, Senior Member, IEEE, and Florent Krzakala Abstract Compressed sensing CS) demonstrates that sparse
More informationEstimation Tasks. Short Course on Image Quality. Matthew A. Kupinski. Introduction
Estimation Tasks Short Course on Image Quality Matthew A. Kupinski Introduction Section 13.3 in B&M Keep in mind the similarities between estimation and classification Image-quality is a statistical concept
More informationLecture Notes 5 Convergence and Limit Theorems. Convergence with Probability 1. Convergence in Mean Square. Convergence in Probability, WLLN
Lecture Notes 5 Convergence and Limit Theorems Motivation Convergence with Probability Convergence in Mean Square Convergence in Probability, WLLN Convergence in Distribution, CLT EE 278: Convergence and
More informationEstimation in Gaussian Noise: Properties of the Minimum Mean-Square Error
Estimation in Gaussian Noise: Properties of the Minimum Mean-Square Error Dongning Guo, Yihong Wu, Shlomo Shamai (Shitz), and Sergio Verdú Abstract arxiv:1004.333v1 [cs.it] 0 Apr 010 Consider the minimum
More informationSparsity Pattern Recovery in Compressed Sensing
Sparsity Pattern Recovery in Compressed Sensing Galen Reeves Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-20-50 http://www.eecs.berkeley.edu/pubs/techrpts/20/eecs-20-50.html
More informationMessage Passing Algorithms for Compressed Sensing: II. Analysis and Validation
Message Passing Algorithms for Compressed Sensing: II. Analysis and Validation David L. Donoho Department of Statistics Arian Maleki Department of Electrical Engineering Andrea Montanari Department of
More informationAsymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
ANALYSIS OF MAP ESTIMATION VIA THE REPLICA METHOD AND APPLICATIONS TO COMPRESSED SENSING Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing Sundeep Rangan,
More informationA Truncated Prediction Framework for Streaming over Erasure Channels
A Truncated Prediction Framework for Streaming over Erasure Channels Farrokh Etezadi, Ashish Khisti, Jun Chen Abstract We propose a new coding technique for sequential transmission of a stream of Gauss-Markov
More informationComputational Methods Short Course on Image Quality
Computational Methods Short Course on Image Quality Matthew A. Kupinski What we Will Cover Sources of randomness Computation of linear-observer performance Computation of ideal-observer performance Miscellaneous
More informationPhase transitions in low-rank matrix estimation
Phase transitions in low-rank matrix estimation Florent Krzakala http://krzakala.github.io/lowramp/ arxiv:1503.00338 & 1507.03857 Thibault Lesieur Lenka Zeborová Caterina Debacco Cris Moore Jess Banks
More informationPlanted Cliques, Iterative Thresholding and Message Passing Algorithms
Planted Cliques, Iterative Thresholding and Message Passing Algorithms Yash Deshpande and Andrea Montanari Stanford University November 5, 2013 Deshpande, Montanari Planted Cliques November 5, 2013 1 /
More informationHamilton-Jacobi-Bellman Equation Feb 25, 2008
Hamilton-Jacobi-Bellman Equation Feb 25, 2008 What is it? The Hamilton-Jacobi-Bellman (HJB) equation is the continuous-time analog to the discrete deterministic dynamic programming algorithm Discrete VS
More informationAnalyticity and Derivatives of Entropy Rate for HMP s
Analyticity and Derivatives of Entropy Rate for HMP s Guangyue Han University of Hong Kong Brian Marcus University of British Columbia 1 Hidden Markov chains A hidden Markov chain Z = {Z i } is a stochastic
More informationEstimating LASSO Risk and Noise Level
Estimating LASSO Risk and Noise Level Mohsen Bayati Stanford University bayati@stanford.edu Murat A. Erdogdu Stanford University erdogdu@stanford.edu Andrea Montanari Stanford University montanar@stanford.edu
More informationAn equivalence between high dimensional Bayes optimal inference and M-estimation
An equivalence between high dimensional Bayes optimal inference and M-estimation Madhu Advani Surya Ganguli Department of Applied Physics, Stanford University msadvani@stanford.edu and sganguli@stanford.edu
More informationStatistical Mechanics of MAP Estimation: General Replica Ansatz
1 Statistical Mechanics of MAP Estimation: General Replica Ansatz Ali Bereyhi, Ralf R. Müller, and Hermann Schulz-Baldes arxiv:1612.01980v2 [cs.it 23 Oct 2017 Abstract The large-system performance of maximum-a-posterior
More informationI. INTRODUCTION. A. Motivation
Approximate Message Passing Algoritm wit Universal Denoising and Gaussian Mixture Learning Yanting Ma, Student Member, IEEE, Junan Zu, Student Member, IEEE, and Dror Baron, Senior Member, IEEE Abstract
More informationSparse Sensing for Statistical Inference
Sparse Sensing for Statistical Inference Model-driven and data-driven paradigms Geert Leus, Sundeep Chepuri, and Georg Kail ITA 2016, 04 Feb 2016 1/17 Power networks, grid analytics Health informatics
More informationGaussian, Markov and stationary processes
Gaussian, Markov and stationary processes Gonzalo Mateos Dept. of ECE and Goergen Institute for Data Science University of Rochester gmateosb@ece.rochester.edu http://www.ece.rochester.edu/~gmateosb/ November
More informationUCSD ECE153 Handout #34 Prof. Young-Han Kim Tuesday, May 27, Solutions to Homework Set #6 (Prepared by TA Fatemeh Arbabjolfaei)
UCSD ECE53 Handout #34 Prof Young-Han Kim Tuesday, May 7, 04 Solutions to Homework Set #6 (Prepared by TA Fatemeh Arbabjolfaei) Linear estimator Consider a channel with the observation Y XZ, where the
More informationVariable-Rate Universal Slepian-Wolf Coding with Feedback
Variable-Rate Universal Slepian-Wolf Coding with Feedback Shriram Sarvotham, Dror Baron, and Richard G. Baraniuk Dept. of Electrical and Computer Engineering Rice University, Houston, TX 77005 Abstract
More informationinformation estimation feedback
relations between information and estimation in the presence of feedback talk at workshop on: Tsachy Weissman Information and Control in Networks LCCC - Lund Center for Control of Complex Engineering Systems
More information11 - The linear model
11-1 The linear model S. Lall, Stanford 2011.02.15.01 11 - The linear model The linear model The joint pdf and covariance Example: uniform pdfs The importance of the prior Linear measurements with Gaussian
More informationGaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y)
Gaussian source Assumptions d = (x-y) 2, given D, find lower bound of I(X;Y) E{(X-Y) 2 } D
More informationFundamental Limits of Compressed Sensing under Optimal Quantization
Fundamental imits of Compressed Sensing under Optimal Quantization Alon Kipnis, Galen Reeves, Yonina C. Eldar and Andrea J. Goldsmith Department of Electrical Engineering, Stanford University Department
More informationStatistical Mechanics of MAP Estimation: General Replica Ansatz
1 Statistical Mechanics of MAP Estimation: General Replica Ansatz Ali Bereyhi, Ralf R. Müller, and Hermann Schulz-Baldes arxiv:1612.01980v1 [cs.it 6 Dec 2016 Abstract The large-system performance of maximum-a-posterior
More informationELEG 5633 Detection and Estimation Minimum Variance Unbiased Estimators (MVUE)
1 ELEG 5633 Detection and Estimation Minimum Variance Unbiased Estimators (MVUE) Jingxian Wu Department of Electrical Engineering University of Arkansas Outline Minimum Variance Unbiased Estimators (MVUE)
More informationMutual information for symmetric rank-one matrix estimation: A proof of the replica formula
Mutual information for symmetric rank-one matrix estimation: A proof of the replica formula Jean Barbier, Mohamad Dia and Nicolas Macris Laboratoire de Théorie des Communications, Faculté Informatique
More informationLecture 14 February 28
EE/Stats 376A: Information Theory Winter 07 Lecture 4 February 8 Lecturer: David Tse Scribe: Sagnik M, Vivek B 4 Outline Gaussian channel and capacity Information measures for continuous random variables
More informationECE531 Lecture 10b: Dynamic Parameter Estimation: System Model
ECE531 Lecture 10b: Dynamic Parameter Estimation: System Model D. Richard Brown III Worcester Polytechnic Institute 02-Apr-2009 Worcester Polytechnic Institute D. Richard Brown III 02-Apr-2009 1 / 14 Introduction
More informationAsymptotic Nonequivalence of Nonparametric Experiments When the Smoothness Index is ½
University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 1998 Asymptotic Nonequivalence of Nonparametric Experiments When the Smoothness Index is ½ Lawrence D. Brown University
More informationCommunication constraints and latency in Networked Control Systems
Communication constraints and latency in Networked Control Systems João P. Hespanha Center for Control Engineering and Computation University of California Santa Barbara In collaboration with Antonio Ortega
More informationOn the Required Accuracy of Transmitter Channel State Information in Multiple Antenna Broadcast Channels
On the Required Accuracy of Transmitter Channel State Information in Multiple Antenna Broadcast Channels Giuseppe Caire University of Southern California Los Angeles, CA, USA Email: caire@usc.edu Nihar
More informationM-Estimation under High-Dimensional Asymptotics
M-Estimation under High-Dimensional Asymptotics 2014-05-01 Classical M-estimation Big Data M-estimation An out-of-the-park grand-slam home run Annals of Mathematical Statistics 1964 Richard Olshen Classical
More informationMidterm Exam Information Theory Fall Midterm Exam. Time: 09:10 12:10 11/23, 2016
Midterm Exam Time: 09:10 12:10 11/23, 2016 Name: Student ID: Policy: (Read before You Start to Work) The exam is closed book. However, you are allowed to bring TWO A4-size cheat sheet (single-sheet, two-sided).
More informationUCSD ECE153 Handout #30 Prof. Young-Han Kim Thursday, May 15, Homework Set #6 Due: Thursday, May 22, 2011
UCSD ECE153 Handout #30 Prof. Young-Han Kim Thursday, May 15, 2014 Homework Set #6 Due: Thursday, May 22, 2011 1. Linear estimator. Consider a channel with the observation Y = XZ, where the signal X and
More informationCoding over Interference Channels: An Information-Estimation View
Coding over Interference Channels: An Information-Estimation View Shlomo Shamai Department of Electrical Engineering Technion - Israel Institute of Technology Information Systems Laboratory Colloquium
More informationEstimation theory. Parametric estimation. Properties of estimators. Minimum variance estimator. Cramer-Rao bound. Maximum likelihood estimators
Estimation theory Parametric estimation Properties of estimators Minimum variance estimator Cramer-Rao bound Maximum likelihood estimators Confidence intervals Bayesian estimation 1 Random Variables Let
More informationTwo Applications of the Gaussian Poincaré Inequality in the Shannon Theory
Two Applications of the Gaussian Poincaré Inequality in the Shannon Theory Vincent Y. F. Tan (Joint work with Silas L. Fong) National University of Singapore (NUS) 2016 International Zurich Seminar on
More informationGraphical Models Concepts in Compressed Sensing
Graphical Models Concepts in Compressed Sensing Andrea Montanari Abstract This paper surveys recent work in applying ideas from graphical models and message passing algorithms to solve large scale regularized
More informationCompressed Sensing Using Bernoulli Measurement Matrices
ITSchool 11, Austin Compressed Sensing Using Bernoulli Measurement Matrices Yuhan Zhou Advisor: Wei Yu Department of Electrical and Computer Engineering University of Toronto, Canada Motivation Motivation
More informationFunctional Properties of MMSE
Functional Properties of MMSE Yihong Wu epartment of Electrical Engineering Princeton University Princeton, NJ 08544, USA Email: yihongwu@princeton.edu Sergio Verdú epartment of Electrical Engineering
More informationSignal Estimation in Gaussian Noise: A Statistical Physics Perspective
Signal Estimation in Gaussian Noise: A Statistical Physics Perspective Neri Merhav Electrical Engineering Dept. Technion Israel Inst. of Tech. Haifa 3000, Israel Email: merhav@ee.technion.ac.il Dongning
More informationCompressed Sensing under Optimal Quantization
Compressed Sensing under Optimal Quantization Alon Kipnis, Galen Reeves, Yonina C. Eldar and Andrea J. Goldsmith Department of Electrical Engineering, Stanford University Department of Electrical and Computer
More informationee378a spring 2013 April 1st intro lecture statistical signal processing/ inference, estimation, and information processing Monday, April 1, 13
ee378a statistical signal processing/ inference, estimation, and information processing spring 2013 April 1st intro lecture 1 what is statistical signal processing? anything & everything inference, estimation,
More informationVector Approximate Message Passing. Phil Schniter
Vector Approximate Message Passing Phil Schniter Collaborators: Sundeep Rangan (NYU), Alyson Fletcher (UCLA) Supported in part by NSF grant CCF-1527162. itwist @ Aalborg University Aug 24, 2016 Standard
More informationConstitutive models: Incremental plasticity Drücker s postulate
Constitutive models: Incremental plasticity Drücker s postulate if consistency condition associated plastic law, associated plasticity - plastic flow law associated with the limit (loading) surface Prager
More informationBinary Classification and Feature Selection via Generalized Approximate Message Passing
Binary Classification and Feature Selection via Generalized Approximate Message Passing Phil Schniter Collaborators: Justin Ziniel (OSU-ECE) and Per Sederberg (OSU-Psych) Supported in part by NSF grant
More informationUCSD ECE250 Handout #20 Prof. Young-Han Kim Monday, February 26, Solutions to Exercise Set #7
UCSD ECE50 Handout #0 Prof. Young-Han Kim Monday, February 6, 07 Solutions to Exercise Set #7. Minimum waiting time. Let X,X,... be i.i.d. exponentially distributed random variables with parameter λ, i.e.,
More information2 Statistical Estimation: Basic Concepts
Technion Israel Institute of Technology, Department of Electrical Engineering Estimation and Identification in Dynamical Systems (048825) Lecture Notes, Fall 2009, Prof. N. Shimkin 2 Statistical Estimation:
More informationSolutions to Homework Set #5 (Prepared by Lele Wang) MSE = E [ (sgn(x) g(y)) 2],, where f X (x) = 1 2 2π e. e (x y)2 2 dx 2π
Solutions to Homework Set #5 (Prepared by Lele Wang). Neural net. Let Y X + Z, where the signal X U[,] and noise Z N(,) are independent. (a) Find the function g(y) that minimizes MSE E [ (sgn(x) g(y))
More informationMultimedia Communications. Scalar Quantization
Multimedia Communications Scalar Quantization Scalar Quantization In many lossy compression applications we want to represent source outputs using a small number of code words. Process of representing
More informationLarge sample covariance matrices and the T 2 statistic
Large sample covariance matrices and the T 2 statistic EURANDOM, the Netherlands Joint work with W. Zhou Outline 1 2 Basic setting Let {X ij }, i, j =, be i.i.d. r.v. Write n s j = (X 1j,, X pj ) T and
More informationStochastic Models (Lecture #4)
Stochastic Models (Lecture #4) Thomas Verdebout Université libre de Bruxelles (ULB) Today Today, our goal will be to discuss limits of sequences of rv, and to study famous limiting results. Convergence
More informationThe Game of Twenty Questions with noisy answers. Applications to Fast face detection, micro-surgical tool tracking and electron microscopy
The Game of Twenty Questions with noisy answers. Applications to Fast face detection, micro-surgical tool tracking and electron microscopy Graduate Summer School: Computer Vision July 22 - August 9, 2013
More informationEconomics 583: Econometric Theory I A Primer on Asymptotics
Economics 583: Econometric Theory I A Primer on Asymptotics Eric Zivot January 14, 2013 The two main concepts in asymptotic theory that we will use are Consistency Asymptotic Normality Intuition consistency:
More informationThreshold Policy for Global Games with Noisy Information Sharing
05 IEEE 54th Annual Conference on Decision and Control CDC December 5-8, 05 Osaka, Japan Threshold olicy for Global Games with Noisy Information Sharing Hessam Mahdavifar, Ahmad Beirami, Behrouz Touri,
More informationAdvanced Signal Processing Introduction to Estimation Theory
Advanced Signal Processing Introduction to Estimation Theory Danilo Mandic, room 813, ext: 46271 Department of Electrical and Electronic Engineering Imperial College London, UK d.mandic@imperial.ac.uk,
More informationInteger-Forcing Source Coding
Or Ordentlich Joint work with Uri Erez June 30th, 2014 ISIT, Honolulu, HI, USA Motivation 1 - Universal Quantization [ x1 x 2 ] N(0,K xx) x 1 E 1 x 2 E 2 D (ˆx 1,d) (ˆx 2,d) Motivation 1 - Universal Quantization
More informationGenerative Models and Stochastic Algorithms for Population Average Estimation and Image Analysis
Generative Models and Stochastic Algorithms for Population Average Estimation and Image Analysis Stéphanie Allassonnière CIS, JHU July, 15th 28 Context : Computational Anatomy Context and motivations :
More information