Finding the best mismatched detector for channel coding and hypothesis testing
|
|
- Blake Dalton
- 5 years ago
- Views:
Transcription
1 Finding the best mismatched detector for channel coding and hypothesis testing Sean Meyn Department of Electrical and Computer Engineering University of Illinois and the Coordinated Science Laboratory Joint work with Emmanuel Abbe, Muriel Me dard, and Lizhong Zheng Laboratory for Information and Decision Systems, MIT Supported in part by Motorola RPS #32 and ITMANET DARPA RK
2 Outline Introduction π 0 P XY Mismatched channel X memoryless Y channel Learning the best detector η ϱ Conclusions β (η ϱ )
3 I Introduction π 0 P XY
4 Background: Mismatch Channel X memoryless channel Y Optimal detector ML: Based on the empirical distributions, i =argmax Γ i n, F i where, F is the log-likelihood ratio (real-valued function of x, y) Γ i n ( i X, Y ) is the empirical distribution of
5 Background: Mismatch Channel X memoryless channel Y Optimal detector ML: Based on the empirical distributions, i =argmax Γ i n, F i where, F is the log-likelihood ratio Γ i n ( i X, Y ) is the empirical distribution of Question considered by Lapidoth 1994, Csiszar et. al. 1981, 1995: What if the channel model is not known exactly?
6 Background: Robust Hypothesis Testing Uncertainty classes defined by moment constraints For given false alarm exponent η, min-max optimal missed detection exponent β P 0, P1 β = inf π 1 P 1 inf D(µ π 1 ) µ Q η (P 0 ) Meyn, Pandit, Veeravalli 200X
7 Background: Robust Hypothesis Testing Uncertainty classes defined by moment constraints For given false alarm exponent η, min-max optimal missed detection exponent β P 0, P1 β = inf π 1 P 1 inf D(µ π 1 ) µ Q η (P 0 ) Q β (P 1 ) π 1 µ π 0 P 1 P 0 µ, F = µ, F Q (P 0 ) η F : Generalized Log Likelihood Ratio Meyn, Pandit, Veeravalli 200X
8 Goal: Robustness Original motivation for the research on mismatched detectors What is the impact on capacity given a poor model? How should codes be constructed to take into account uncertainty? X memoryless Y channel
9 Goal: Complexity Detection and capacity - easy. Its just mutual information!! Linear detectors suggest relaxation techniques for multiuser and network settings
10 Goal: Adaptation A new point of view! How can a detector be tuned on-line in a dynamic environment? X memoryless Y channel This will depend on SNR Other users Network conditions... Estimates Capacity
11 II Mismatched Channel X memoryless Y channel
12 Mismatched Detector A given function F (x,y) defines a surrogate ML detector: i =argmax i Empirical distributions for ith codeword: Γ i n = n 1 Γ i n, F =argmax i Reliably received rate (in general a lower bound): Generalized Mutual Information (Lapidoth, Csiszar) n t=1 δ X i t,y t n t=1 F (X i t, Y t )
13 Mismatched Detector: Sanov s Theorem Sanov s Theorem (in form of Chernoff s Bound) gives log P e ni LDP (P X ; F ) ( Pe pair-wise prob. of error) { } I LDP (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F
14 { Mismatched Detector: Sanov s Theorem Sanov s Theorem (in form of Chernoff s Bound) gives log P e ni LDP (P X ; F ) ( Pe pair-wise prob. of error) { } I LDP (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F π 0 Q β0 (π 0 )={ Γ : D(Γ ) β 0 π 0 = P X P Y π 0
15 { Mismatched Detector: Sanov s Theorem Sanov s Theorem (in form of Chernoff s Bound) gives log P e ni LDP (P X ; F ) { } I LDP (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F π 0 Q β0 (π 0 )={ Γ : D(Γ ) β 0 π 0 = P X P Y π 0 Γ, F = P XY, F P XY β 0 = I LDP (P X ; F )
16 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information - introduces typicality constraint, log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y }
17 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y } Derivation of GMI bound using Sanov:
18 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y } Derivation of GMI bound using Sanov: For any function G(y), i =argmax i { n } Γ i n, F + G =argmax [F (Xt, i Y t )+G(Y t )] i t=1 log P e ni LDP (P X ; F + G) any G(y)
19 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y } log P e ni LDP (P X ; F + G) any G(y) sup I LDP (P X ; F + G) =I GMI (P X ; F ) G(y) G * : Lagrange multiplier for equality constraint Γ 2 = P Y
20 III Learning the Best Detector η ϱ β (η ϱ )
21 Learning the Best Test Given a parameterized class of detectors, What is the best value of α? {F α = α i ψ i : α R d } Maximize MM capacity: J(α):=I LDP (P X ; F α )
22 Learning the Best Test Maximize MM capacity: J(α):=I LDP (P X ; F α ) Derivatives: J = ψ Γ α, ψ 2 J = Γ α, ψψ T Γ α, ψ Γ α, ψ T where for any set A B(X Y), Γ α (A):= P X P Y, e F α I A P X P Y, e F α
23 Learning the Best Test Maximize MM capacity: Derivatives: J(α):=I LDP (P X ; F α ) J = ψ Γ α, ψ 2 J = Γ α, ψψ T Γ α, ψ Γ α, ψ T where for any set A B(X Y), Γ α (A):= P X P Y, e F α I A P X P Y, e F α Newton Raphson based on channel model, or Stochastic Newton Raphson based on measurements alone!
24 Learning the Best Test Estimation algorithm is decentralized and model-free X memoryless Y channel X Estimation α(t)
25 AWGN Channel Y = X + N SNR = Basis: ψ =(x 2, y 2, xy) T Estimates Capacity iterations
26 Mismatched Neyman Pearson Hypothesis Testing Numerical experiment: X 1 uniform on [0, 1], X 0 = 5 X 1 η β (η) False Alarm Error Exponent Missed Detection Error Exponent Stochastic Newton-Raphson to find [ η, β (η) ] pairs Parameterized class of detectors, {F α = α i ψ i : α R d } ψ(x) =[x,log(x)/10] T Optimal detector
27 Mismatched Neyman Pearson Hypothesis Testing X uniform on [0, 1], X 0 = 5 X 1 Stochastic Newton-Raphson η ϱ False Alarm Error Exponent β (η ϱ ) Missed Detection Error Exponent Parallel simulation for a range of η
28 Mismatched Neyman Pearson Hypothesis Testing X 1 uniform on [0, 1], X 0 = 5 X 1 β (η) ψ(x) =[x,log(x)/10] T ψ(x) =[x,x, 2..., x 6 ] T ψ(x) =[x, (1 + 10x) 4] ψ(x) =[x,x 2 ] T ψ(x) =[x,1/(1 + 10x)] T Optimal detector T 0.10 D(π 1 π 0 )= Parallel simulation for a range of η 2.4 η
29 IV Conclusions
30 Conclusions Geometry based on Sanov s Theorem combined with Stochastic Approximation provides powerful computational tools
31 Conclusions Geometry based on Sanov s Theorem combined with Stochastic Approximation provides powerful computational tools Future work: Optimizing the input distribution Application to MIMO will simultaneously resolve coding and resource allocation. Extensions to network coding possible? Simulation algorithm exhibits high variance. Application of importance sampling? Revisit robust approach?
32 References [1] E. Abbe, M. Médard, S. Meyn, and L. Zheng. Geometry of mismatched decoders. Submitted to IEEE International Symposium on Information Theory, [3] S. Borade and L. Zheng. I-projection and the geometry of error exponents. In Proceedings of the Forty-Fourth Annual Allerton Conference on Communication, Control, and Computing, Sept 27-29, 2006, UIUC, Illinois, USA, [4] I. Csiszár. The method of types. IEEE Trans. Inform. Theory, 44(6): , Information theory: [7] I. Csiszár and P. Narayan. Channel capacity for a given decoding metric. IEEE Trans. Inform. Theory, 41(1):35 43, [12] G. Kaplan, A. Lapidoth, S. Shamai, and N. Merhav. On information rates for mismatched decoders. IEEE Trans. Inform. Theory, 40(6): , [14] C. Pandit and S. P. Meyn. Worst-case large-deviations with application to queueing and information theory. Stoch. Proc. Applns., 116(5): , May [15] O. Zeitouni and M. Gutman. On universal hypotheses testing via large deviations. IEEE Trans. Inform. Theory, 37(2): , 1991.
Entropy, Inference, and Channel Coding
Entropy, Inference, and Channel Coding Sean Meyn Department of Electrical and Computer Engineering University of Illinois and the Coordinated Science Laboratory NSF support: ECS 02-17836, ITR 00-85929
More informationMismatched Multi-letter Successive Decoding for the Multiple-Access Channel
Mismatched Multi-letter Successive Decoding for the Multiple-Access Channel Jonathan Scarlett University of Cambridge jms265@cam.ac.uk Alfonso Martinez Universitat Pompeu Fabra alfonso.martinez@ieee.org
More informationUncertainty. Jayakrishnan Unnikrishnan. CSL June PhD Defense ECE Department
Decision-Making under Statistical Uncertainty Jayakrishnan Unnikrishnan PhD Defense ECE Department University of Illinois at Urbana-Champaign CSL 141 12 June 2010 Statistical Decision-Making Relevant in
More informationAn Achievable Error Exponent for the Mismatched Multiple-Access Channel
An Achievable Error Exponent for the Mismatched Multiple-Access Channel Jonathan Scarlett University of Cambridge jms265@camacuk Albert Guillén i Fàbregas ICREA & Universitat Pompeu Fabra University of
More informationThe Method of Types and Its Application to Information Hiding
The Method of Types and Its Application to Information Hiding Pierre Moulin University of Illinois at Urbana-Champaign www.ifp.uiuc.edu/ moulin/talks/eusipco05-slides.pdf EUSIPCO Antalya, September 7,
More informationUniversal A Posteriori Metrics Game
1 Universal A Posteriori Metrics Game Emmanuel Abbe LCM, EPFL Lausanne, 1015, Switzerland emmanuel.abbe@epfl.ch Rethnakaran Pulikkoonattu Broadcom Inc, Irvine, CA, USA, 92604 rethna@broadcom.com arxiv:1004.4815v1
More informationIEEE Proof Web Version
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 55, NO. 10, OCTOBER 2009 1 A Neyman Pearson Approach to Universal Erasure List Decoding Pierre Moulin, Fellow, IEEE Abstract When information is to be transmitted
More informationStatistical Modeling and Analysis of Content Identification
Statistical Modeling and Analysis of Content Identification Pierre Moulin University of Illinois Beckman Inst., Coord. Sci. ab., & ECE Dept. 405 N. Mathews Ave., Urbana, I 680, USA Email: moulin@ifp.uiuc.edu
More informationThe Poisson Channel with Side Information
The Poisson Channel with Side Information Shraga Bross School of Enginerring Bar-Ilan University, Israel brosss@macs.biu.ac.il Amos Lapidoth Ligong Wang Signal and Information Processing Laboratory ETH
More informationOn the Optimality of Treating Interference as Noise in Competitive Scenarios
On the Optimality of Treating Interference as Noise in Competitive Scenarios A. DYTSO, D. TUNINETTI, N. DEVROYE WORK PARTIALLY FUNDED BY NSF UNDER AWARD 1017436 OUTLINE CHANNEL MODEL AND PAST WORK ADVANTAGES
More informationLecture 22: Error exponents in hypothesis testing, GLRT
10-704: Information Processing and Learning Spring 2012 Lecture 22: Error exponents in hypothesis testing, GLRT Lecturer: Aarti Singh Scribe: Aarti Singh Disclaimer: These notes have not been subjected
More informationOptimal Power Control in Decentralized Gaussian Multiple Access Channels
1 Optimal Power Control in Decentralized Gaussian Multiple Access Channels Kamal Singh Department of Electrical Engineering Indian Institute of Technology Bombay. arxiv:1711.08272v1 [eess.sp] 21 Nov 2017
More informationSHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe
SHARED INFORMATION Prakash Narayan with Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe 2/41 Outline Two-terminal model: Mutual information Operational meaning in: Channel coding: channel
More informationLarge-Deviations and Applications for Learning Tree-Structured Graphical Models
Large-Deviations and Applications for Learning Tree-Structured Graphical Models Vincent Tan Stochastic Systems Group, Lab of Information and Decision Systems, Massachusetts Institute of Technology Thesis
More informationTightened Upper Bounds on the ML Decoding Error Probability of Binary Linear Block Codes and Applications
on the ML Decoding Error Probability of Binary Linear Block Codes and Moshe Twitto Department of Electrical Engineering Technion-Israel Institute of Technology Haifa 32000, Israel Joint work with Igal
More informationAnalytical Bounds on Maximum-Likelihood Decoded Linear Codes: An Overview
Analytical Bounds on Maximum-Likelihood Decoded Linear Codes: An Overview Igal Sason Department of Electrical Engineering, Technion Haifa 32000, Israel Sason@ee.technion.ac.il December 21, 2004 Background
More informationLeast Favorable Distributions for Robust Quickest Change Detection
Least Favorable Distributions for Robust Quickest hange Detection Jayakrishnan Unnikrishnan, Venugopal V. Veeravalli, Sean Meyn Department of Electrical and omputer Engineering, and oordinated Science
More informationError Exponent Region for Gaussian Broadcast Channels
Error Exponent Region for Gaussian Broadcast Channels Lihua Weng, S. Sandeep Pradhan, and Achilleas Anastasopoulos Electrical Engineering and Computer Science Dept. University of Michigan, Ann Arbor, MI
More informationGeneralized Neyman Pearson optimality of empirical likelihood for testing parameter hypotheses
Ann Inst Stat Math (2009) 61:773 787 DOI 10.1007/s10463-008-0172-6 Generalized Neyman Pearson optimality of empirical likelihood for testing parameter hypotheses Taisuke Otsu Received: 1 June 2007 / Revised:
More informationIEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 57, NO. 3, MARCH
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 57, NO 3, MARCH 2011 1587 Universal and Composite Hypothesis Testing via Mismatched Divergence Jayakrishnan Unnikrishnan, Member, IEEE, Dayu Huang, Student
More informationCensoring for Type-Based Multiple Access Scheme in Wireless Sensor Networks
Censoring for Type-Based Multiple Access Scheme in Wireless Sensor Networks Mohammed Karmoose Electrical Engineering Department Alexandria University Alexandria 1544, Egypt Email: mhkarmoose@ieeeorg Karim
More informationUniversal Anytime Codes: An approach to uncertain channels in control
Universal Anytime Codes: An approach to uncertain channels in control paper by Stark Draper and Anant Sahai presented by Sekhar Tatikonda Wireless Foundations Department of Electrical Engineering and Computer
More informationBit-Interleaved Coded Modulation Revisited: A Mismatched Decoding Perspective
Bit-Interleaved Coded Modulation Revisited: A Mismatched Decoding Perspective Alfonso Martinez, Albert Guillén i Fàbregas, Giuseppe Caire and Frans Willems Abstract arxiv:0805.37v [cs.it] 9 May 008 We
More informationFully-distributed spectrum sensing: application to cognitive radio
Fully-distributed spectrum sensing: application to cognitive radio Philippe Ciblat Dpt Comelec, Télécom ParisTech Joint work with F. Iutzeler (PhD student funded by DGA grant) Cognitive radio principle
More informationOptimal Distributed Detection Strategies for Wireless Sensor Networks
Optimal Distributed Detection Strategies for Wireless Sensor Networks Ke Liu and Akbar M. Sayeed University of Wisconsin-Madison kliu@cae.wisc.edu, akbar@engr.wisc.edu Abstract We study optimal distributed
More informationSHARED INFORMATION. Prakash Narayan with. Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe
SHARED INFORMATION Prakash Narayan with Imre Csiszár, Sirin Nitinawarat, Himanshu Tyagi, Shun Watanabe 2/40 Acknowledgement Praneeth Boda Himanshu Tyagi Shun Watanabe 3/40 Outline Two-terminal model: Mutual
More informationChapter 2 Signal Processing at Receivers: Detection Theory
Chapter Signal Processing at Receivers: Detection Theory As an application of the statistical hypothesis testing, signal detection plays a key role in signal processing at receivers of wireless communication
More informationBit-interleaved coded modulation revisited: a mismatched decoding perspective Martinez Vicente, A.; Guillén i Fàbregas, A.; Caire, G.; Willems, F.M.J.
Bit-interleaved coded modulation revisited: a mismatched decoding perspective Martinez Vicente, A.; Guillén i Fàbregas, A.; Caire, G.; Willems, F.M.J. Published in: IEEE International Symposium on Information
More informationINFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS. Michael A. Lexa and Don H. Johnson
INFORMATION PROCESSING ABILITY OF BINARY DETECTORS AND BLOCK DECODERS Michael A. Lexa and Don H. Johnson Rice University Department of Electrical and Computer Engineering Houston, TX 775-892 amlexa@rice.edu,
More informationLecture 7. Union bound for reducing M-ary to binary hypothesis testing
Lecture 7 Agenda for the lecture M-ary hypothesis testing and the MAP rule Union bound for reducing M-ary to binary hypothesis testing Introduction of the channel coding problem 7.1 M-ary hypothesis testing
More informationTD Learning. Sean Meyn. Department of Electrical and Computer Engineering University of Illinois & the Coordinated Science Laboratory
TD Learning Sean Meyn Department of Electrical and Computer Engineering University of Illinois & the Coordinated Science Laboratory Joint work with I.-K. Cho, Illinois S. Mannor, McGill V. Tadic, Sheffield
More informationShannon meets Wiener II: On MMSE estimation in successive decoding schemes
Shannon meets Wiener II: On MMSE estimation in successive decoding schemes G. David Forney, Jr. MIT Cambridge, MA 0239 USA forneyd@comcast.net Abstract We continue to discuss why MMSE estimation arises
More informationInformation Theory and Hypothesis Testing
Summer School on Game Theory and Telecommunications Campione, 7-12 September, 2014 Information Theory and Hypothesis Testing Mauro Barni University of Siena September 8 Review of some basic results linking
More informationEmpirical Likelihood Ratio Test with Distribution Function Constraints
PAPER DRAFT Empirical Likelihood Ratio Test with Distribution Function Constraints Yingxi Liu, Student member, IEEE, Ahmed Tewfik, Fellow, IEEE arxiv:65.57v [math.st] 3 Apr 26 Abstract In this work, we
More informationUniversal and Composite Hypothesis Testing via Mismatched Divergence
Universal and Composite Hypothesis Testing via Mismatched Divergence Jayakrishnan Unnikrishnan, Dayu Huang, Sean Meyn, Amit Surana and Venugopal Veeravalli Abstract arxiv:0909.2234v1 [cs.it] 11 Sep 2009
More informationOn Scalable Source Coding for Multiple Decoders with Side Information
On Scalable Source Coding for Multiple Decoders with Side Information Chao Tian School of Computer and Communication Sciences Laboratory for Information and Communication Systems (LICOS), EPFL, Lausanne,
More informationTime-division multiplexing for green broadcasting
Time-division multiplexing for green broadcasting Pulkit Grover and Anant Sahai Wireless Foundations, Department of EECS University of California at Berkeley Email: {pulkit, sahai} @ eecs.berkeley.edu
More informationUniversal Outlier Hypothesis Testing
Universal Outlier Hypothesis Testing Venu Veeravalli ECE Dept & CSL & ITI University of Illinois at Urbana-Champaign http://www.ifp.illinois.edu/~vvv (with Sirin Nitinawarat and Yun Li) NSF Workshop on
More information2. What are the tradeoffs among different measures of error (e.g. probability of false alarm, probability of miss, etc.)?
ECE 830 / CS 76 Spring 06 Instructors: R. Willett & R. Nowak Lecture 3: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we
More informationIN MOST problems pertaining to hypothesis testing, one
1602 IEEE SIGNAL PROCESSING LETTERS, VOL. 23, NO. 11, NOVEMBER 2016 Joint Detection and Decoding in the Presence of Prior Information With Uncertainty Suat Bayram, Member, IEEE, Berkan Dulek, Member, IEEE,
More informationLecture 1: The Multiple Access Channel. Copyright G. Caire 12
Lecture 1: The Multiple Access Channel Copyright G. Caire 12 Outline Two-user MAC. The Gaussian case. The K-user case. Polymatroid structure and resource allocation problems. Copyright G. Caire 13 Two-user
More informationLecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics. 1 Executive summary
ECE 830 Spring 207 Instructor: R. Willett Lecture 5: Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics Executive summary In the last lecture we saw that the likelihood
More informationCapacity and Reliability Function for Small Peak Signal Constraints
828 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 48, NO. 4, APRIL 2002 Capacity and Reliability Function for Small Peak Signal Constraints Bruce Hajek, Fellow, IEEE, and Vijay G. Subramanian, Member,
More informationInteractions of Information Theory and Estimation in Single- and Multi-user Communications
Interactions of Information Theory and Estimation in Single- and Multi-user Communications Dongning Guo Department of Electrical Engineering Princeton University March 8, 2004 p 1 Dongning Guo Communications
More informationAppendix B Information theory from first principles
Appendix B Information theory from first principles This appendix discusses the information theory behind the capacity expressions used in the book. Section 8.3.4 is the only part of the book that supposes
More informationDecomposition Methods for Large Scale LP Decoding
Decomposition Methods for Large Scale LP Decoding Siddharth Barman Joint work with Xishuo Liu, Stark Draper, and Ben Recht Outline Background and Problem Setup LP Decoding Formulation Optimization Framework
More informationAn Extended Fano s Inequality for the Finite Blocklength Coding
An Extended Fano s Inequality for the Finite Bloclength Coding Yunquan Dong, Pingyi Fan {dongyq8@mails,fpy@mail}.tsinghua.edu.cn Department of Electronic Engineering, Tsinghua University, Beijing, P.R.
More informationTightened Upper Bounds on the ML Decoding Error Probability of Binary Linear Block Codes and Applications
on the ML Decoding Error Probability of Binary Linear Block Codes and Department of Electrical Engineering Technion-Israel Institute of Technology An M.Sc. Thesis supervisor: Dr. Igal Sason March 30, 2006
More informationErgodic and Outage Capacity of Narrowband MIMO Gaussian Channels
Ergodic and Outage Capacity of Narrowband MIMO Gaussian Channels Yang Wen Liang Department of Electrical and Computer Engineering The University of British Columbia April 19th, 005 Outline of Presentation
More informationThe Information Bottleneck Revisited or How to Choose a Good Distortion Measure
The Information Bottleneck Revisited or How to Choose a Good Distortion Measure Peter Harremoës Centrum voor Wiskunde en Informatica PO 94079, 1090 GB Amsterdam The Nederlands PHarremoes@cwinl Naftali
More informationELEC546 Review of Information Theory
ELEC546 Review of Information Theory Vincent Lau 1/1/004 1 Review of Information Theory Entropy: Measure of uncertainty of a random variable X. The entropy of X, H(X), is given by: If X is a discrete random
More informationAn Alternative Proof of the Rate-Distortion Function of Poisson Processes with a Queueing Distortion Measure
An Alternative Proof of the Rate-Distortion Function of Poisson Processes with a Queueing Distortion Measure Todd P. Coleman, Negar Kiyavash, and Vijay G. Subramanian ECE Dept & Coordinated Science Laboratory,
More informationFeedback Capacity of the Compound Channel
Feedback Capacity of the Compound Channel The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation As Published Publisher Shrader,
More informationInformation Theory Meets Game Theory on The Interference Channel
Information Theory Meets Game Theory on The Interference Channel Randall A. Berry Dept. of EECS Northwestern University e-mail: rberry@eecs.northwestern.edu David N. C. Tse Wireless Foundations University
More informationChannel Dispersion and Moderate Deviations Limits for Memoryless Channels
Channel Dispersion and Moderate Deviations Limits for Memoryless Channels Yury Polyanskiy and Sergio Verdú Abstract Recently, Altug and Wagner ] posed a question regarding the optimal behavior of the probability
More informationCooperative Spectrum Sensing for Cognitive Radios under Bandwidth Constraints
Cooperative Spectrum Sensing for Cognitive Radios under Bandwidth Constraints Chunhua Sun, Wei Zhang, and haled Ben Letaief, Fellow, IEEE Department of Electronic and Computer Engineering The Hong ong
More informationLecture 8: Information Theory and Statistics
Lecture 8: Information Theory and Statistics Part II: Hypothesis Testing and I-Hsiang Wang Department of Electrical Engineering National Taiwan University ihwang@ntu.edu.tw December 23, 2015 1 / 50 I-Hsiang
More informationOptimization in Information Theory
Optimization in Information Theory Dawei Shen November 11, 2005 Abstract This tutorial introduces the application of optimization techniques in information theory. We revisit channel capacity problem from
More information10-704: Information Processing and Learning Fall Lecture 24: Dec 7
0-704: Information Processing and Learning Fall 206 Lecturer: Aarti Singh Lecture 24: Dec 7 Note: These notes are based on scribed notes from Spring5 offering of this course. LaTeX template courtesy of
More information(each row defines a probability distribution). Given n-strings x X n, y Y n we can use the absence of memory in the channel to compute
ENEE 739C: Advanced Topics in Signal Processing: Coding Theory Instructor: Alexander Barg Lecture 6 (draft; 9/6/03. Error exponents for Discrete Memoryless Channels http://www.enee.umd.edu/ abarg/enee739c/course.html
More information2.1.3 The Testing Problem and Neave s Step Method
we can guarantee (1) that the (unknown) true parameter vector θ t Θ is an interior point of Θ, and (2) that ρ θt (R) > 0 for any R 2 Q. These are two of Birch s regularity conditions that were critical
More informationCell throughput analysis of the Proportional Fair scheduler in the single cell environment
Cell throughput analysis of the Proportional Fair scheduler in the single cell environment Jin-Ghoo Choi and Seawoong Bahk IEEE Trans on Vehicular Tech, Mar 2007 *** Presented by: Anh H. Nguyen February
More informationRandom Matrices and Wireless Communications
Random Matrices and Wireless Communications Jamie Evans Centre for Ultra-Broadband Information Networks (CUBIN) Department of Electrical and Electronic Engineering University of Melbourne 3.5 1 3 0.8 2.5
More informationRandom Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws. Symeon Chatzinotas February 11, 2013 Luxembourg
Random Matrix Theory Lecture 1 Introduction, Ensembles and Basic Laws Symeon Chatzinotas February 11, 2013 Luxembourg Outline 1. Random Matrix Theory 1. Definition 2. Applications 3. Asymptotics 2. Ensembles
More informationOn the Duality between Multiple-Access Codes and Computation Codes
On the Duality between Multiple-Access Codes and Computation Codes Jingge Zhu University of California, Berkeley jingge.zhu@berkeley.edu Sung Hoon Lim KIOST shlim@kiost.ac.kr Michael Gastpar EPFL michael.gastpar@epfl.ch
More informationJoint FEC Encoder and Linear Precoder Design for MIMO Systems with Antenna Correlation
Joint FEC Encoder and Linear Precoder Design for MIMO Systems with Antenna Correlation Chongbin Xu, Peng Wang, Zhonghao Zhang, and Li Ping City University of Hong Kong 1 Outline Background Mutual Information
More informationUniversal and Composite Hypothesis Testing via Mismatched Divergence
1 Universal and Composite Hypothesis Testing via Mismatched Divergence arxiv:0909.2234v3 [cs.it] 9 Sep 2010 Jayakrishnan Unnikrishnan, Dayu Huang, Sean Meyn, Amit Surana and Venugopal Veeravalli Abstract
More informationAverage-cost temporal difference learning and adaptive control variates
Average-cost temporal difference learning and adaptive control variates Sean Meyn Department of ECE and the Coordinated Science Laboratory Joint work with S. Mannor, McGill V. Tadic, Sheffield S. Henderson,
More informationOn the Joint Decoding of LDPC Codes and Finite-State Channels via Linear Programming
On the Joint Decoding of LDPC Codes and Finite-State Channels via Linear Programming Byung-Hak Kim (joint with Henry D. Pfister) Texas A&M University College Station International Symposium on Information
More informationA Power Efficient Sensing/Communication Scheme: Joint Source-Channel-Network Coding by Using Compressive Sensing
Forty-Ninth Annual Allerton Conference Allerton House, UIUC, Illinois, USA September 28-30, 20 A Power Efficient Sensing/Communication Scheme: Joint Source-Channel-Network Coding by Using Compressive Sensing
More informationUniversity of Siena. Multimedia Security. Watermark extraction. Mauro Barni University of Siena. M. Barni, University of Siena
Multimedia Security Mauro Barni University of Siena : summary Optimum decoding/detection Additive SS watermarks Decoding/detection of QIM watermarks The dilemma of de-synchronization attacks Geometric
More informationExact Random Coding Error Exponents of Optimal Bin Index Decoding
Exact Random Coding Error Exponents of Optimal Bin Index Decoding Neri Merhav Department of Electrical Engineering Technion - Israel Institute of Technology Technion City, Haifa 32000, ISRAEL E mail: merhav@ee.technion.ac.il
More informationEE 4TM4: Digital Communications II. Channel Capacity
EE 4TM4: Digital Communications II 1 Channel Capacity I. CHANNEL CODING THEOREM Definition 1: A rater is said to be achievable if there exists a sequence of(2 nr,n) codes such thatlim n P (n) e (C) = 0.
More informationOn the Minimum Mean p-th Error in Gaussian Noise Channels and its Applications
On the Minimum Mean p-th Error in Gaussian Noise Channels and its Applications Alex Dytso, Ronit Bustin, Daniela Tuninetti, Natasha Devroye, H. Vincent Poor, and Shlomo Shamai (Shitz) Notation X 2 R n,
More informationCONSIDER a joint stationary and memoryless process
4006 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 55, NO 9, SEPTEMBER 2009 On the Duality Between Slepian Wolf Coding and Channel Coding Under Mismatched Decoding Jun Chen, Member, IEEE, Da-ke He, and
More informationAchievable Rates for Probabilistic Shaping
Achievable Rates for Probabilistic Shaping Georg Böcherer Mathematical and Algorithmic Sciences Lab Huawei Technologies France S.A.S.U. georg.boecherer@ieee.org May 3, 08 arxiv:707.034v5 cs.it May 08 For
More informationOptimal Power Allocation over Parallel Gaussian Broadcast Channels
Optimal Power Allocation over Parallel Gaussian Broadcast Channels David N.C. Tse Dept. of Electrical Engineering and Computer Sciences University of California at Berkeley email: dtse@eecs.berkeley.edu
More informationANALYSIS OF A PARTIAL DECORRELATOR IN A MULTI-CELL DS/CDMA SYSTEM
ANAYSIS OF A PARTIA DECORREATOR IN A MUTI-CE DS/CDMA SYSTEM Mohammad Saquib ECE Department, SU Baton Rouge, A 70803-590 e-mail: saquib@winlab.rutgers.edu Roy Yates WINAB, Rutgers University Piscataway
More informationg(.) 1/ N 1/ N Decision Decision Device u u u u CP
Distributed Weak Signal Detection and Asymptotic Relative Eciency in Dependent Noise Hakan Delic Signal and Image Processing Laboratory (BUSI) Department of Electrical and Electronics Engineering Bogazici
More informationOn the Secrecy Capacity of the Z-Interference Channel
On the Secrecy Capacity of the Z-Interference Channel Ronit Bustin Tel Aviv University Email: ronitbustin@post.tau.ac.il Mojtaba Vaezi Princeton University Email: mvaezi@princeton.edu Rafael F. Schaefer
More informationEECS 750. Hypothesis Testing with Communication Constraints
EECS 750 Hypothesis Testing with Communication Constraints Name: Dinesh Krithivasan Abstract In this report, we study a modification of the classical statistical problem of bivariate hypothesis testing.
More informationCorrelation Detection and an Operational Interpretation of the Rényi Mutual Information
Correlation Detection and an Operational Interpretation of the Rényi Mutual Information Masahito Hayashi 1, Marco Tomamichel 2 1 Graduate School of Mathematics, Nagoya University, and Centre for Quantum
More informationWHEN IS A MAXIMAL INVARIANT HYPOTHESIS TEST BETTER THAN THE GLRT? Hyung Soo Kim and Alfred O. Hero
WHEN IS A MAXIMAL INVARIANT HYPTHESIS TEST BETTER THAN THE GLRT? Hyung Soo Kim and Alfred. Hero Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor, MI 489-222 ABSTRACT
More informationThe Capacity of the Semi-Deterministic Cognitive Interference Channel and its Application to Constant Gap Results for the Gaussian Channel
The Capacity of the Semi-Deterministic Cognitive Interference Channel and its Application to Constant Gap Results for the Gaussian Channel Stefano Rini, Daniela Tuninetti, and Natasha Devroye Department
More informationOn Information Divergence Measures, Surrogate Loss Functions and Decentralized Hypothesis Testing
On Information Divergence Measures, Surrogate Loss Functions and Decentralized Hypothesis Testing XuanLong Nguyen Martin J. Wainwright Michael I. Jordan Electrical Engineering & Computer Science Department
More informationIN HYPOTHESIS testing problems, a decision-maker aims
IEEE SIGNAL PROCESSING LETTERS, VOL. 25, NO. 12, DECEMBER 2018 1845 On the Optimality of Likelihood Ratio Test for Prospect Theory-Based Binary Hypothesis Testing Sinan Gezici, Senior Member, IEEE, and
More informationEnergy State Amplification in an Energy Harvesting Communication System
Energy State Amplification in an Energy Harvesting Communication System Omur Ozel Sennur Ulukus Department of Electrical and Computer Engineering University of Maryland College Park, MD 20742 omur@umd.edu
More informationMultiuser Capacity in Block Fading Channel
Multiuser Capacity in Block Fading Channel April 2003 1 Introduction and Model We use a block-fading model, with coherence interval T where M independent users simultaneously transmit to a single receiver
More informationStrong Converse Theorems for Classes of Multimessage Multicast Networks: A Rényi Divergence Approach
Strong Converse Theorems for Classes of Multimessage Multicast Networks: A Rényi Divergence Approach Silas Fong (Joint work with Vincent Tan) Department of Electrical & Computer Engineering National University
More informationA Proof of the Converse for the Capacity of Gaussian MIMO Broadcast Channels
A Proof of the Converse for the Capacity of Gaussian MIMO Broadcast Channels Mehdi Mohseni Department of Electrical Engineering Stanford University Stanford, CA 94305, USA Email: mmohseni@stanford.edu
More informationECE531 Screencast 9.2: N-P Detection with an Infinite Number of Possible Observations
ECE531 Screencast 9.2: N-P Detection with an Infinite Number of Possible Observations D. Richard Brown III Worcester Polytechnic Institute Worcester Polytechnic Institute D. Richard Brown III 1 / 7 Neyman
More informationSuperposition Encoding and Partial Decoding Is Optimal for a Class of Z-interference Channels
Superposition Encoding and Partial Decoding Is Optimal for a Class of Z-interference Channels Nan Liu and Andrea Goldsmith Department of Electrical Engineering Stanford University, Stanford CA 94305 Email:
More informationDispersion of the Gilbert-Elliott Channel
Dispersion of the Gilbert-Elliott Channel Yury Polyanskiy Email: ypolyans@princeton.edu H. Vincent Poor Email: poor@princeton.edu Sergio Verdú Email: verdu@princeton.edu Abstract Channel dispersion plays
More informationConvexity/Concavity of Renyi Entropy and α-mutual Information
Convexity/Concavity of Renyi Entropy and -Mutual Information Siu-Wai Ho Institute for Telecommunications Research University of South Australia Adelaide, SA 5095, Australia Email: siuwai.ho@unisa.edu.au
More informationDynamic spectrum access with learning for cognitive radio
1 Dynamic spectrum access with learning for cognitive radio Jayakrishnan Unnikrishnan and Venugopal V. Veeravalli Department of Electrical and Computer Engineering, and Coordinated Science Laboratory University
More informationVariable Length Codes for Degraded Broadcast Channels
Variable Length Codes for Degraded Broadcast Channels Stéphane Musy School of Computer and Communication Sciences, EPFL CH-1015 Lausanne, Switzerland Email: stephane.musy@ep.ch Abstract This paper investigates
More informationBelief Propagation, Information Projections, and Dykstra s Algorithm
Belief Propagation, Information Projections, and Dykstra s Algorithm John MacLaren Walsh, PhD Department of Electrical and Computer Engineering Drexel University Philadelphia, PA jwalsh@ece.drexel.edu
More information3052 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 7, JULY /$ IEEE
3052 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL 52, NO 7, JULY 2006 Capacity of Nearly Decomposable Markovian Fading Channels Under Asymmetric Receiver Sender Side Information Muriel Médard, Senior Member,
More informationDecentralized Detection In Wireless Sensor Networks
Decentralized Detection In Wireless Sensor Networks Milad Kharratzadeh Department of Electrical & Computer Engineering McGill University Montreal, Canada April 2011 Statistical Detection and Estimation
More information