Finding the best mismatched detector for channel coding and hypothesis testing

Transcription

1 Finding the best mismatched detector for channel coding and hypothesis testing Sean Meyn Department of Electrical and Computer Engineering University of Illinois and the Coordinated Science Laboratory Joint work with Emmanuel Abbe, Muriel Me dard, and Lizhong Zheng Laboratory for Information and Decision Systems, MIT Supported in part by Motorola RPS #32 and ITMANET DARPA RK

2 Outline Introduction π 0 P XY Mismatched channel X memoryless Y channel Learning the best detector η ϱ Conclusions β (η ϱ )

3 I Introduction π 0 P XY

4 Background: Mismatch Channel X memoryless channel Y Optimal detector ML: Based on the empirical distributions, i =argmax Γ i n, F i where, F is the log-likelihood ratio (real-valued function of x, y) Γ i n ( i X, Y ) is the empirical distribution of

5 Background: Mismatch Channel X memoryless channel Y Optimal detector ML: Based on the empirical distributions, i =argmax Γ i n, F i where, F is the log-likelihood ratio Γ i n ( i X, Y ) is the empirical distribution of Question considered by Lapidoth 1994, Csiszar et. al. 1981, 1995: What if the channel model is not known exactly?

6 Background: Robust Hypothesis Testing Uncertainty classes defined by moment constraints For given false alarm exponent η, min-max optimal missed detection exponent β P 0, P1 β = inf π 1 P 1 inf D(µ π 1 ) µ Q η (P 0 ) Meyn, Pandit, Veeravalli 200X

7 Background: Robust Hypothesis Testing Uncertainty classes defined by moment constraints For given false alarm exponent η, min-max optimal missed detection exponent β P 0, P1 β = inf π 1 P 1 inf D(µ π 1 ) µ Q η (P 0 ) Q β (P 1 ) π 1 µ π 0 P 1 P 0 µ, F = µ, F Q (P 0 ) η F : Generalized Log Likelihood Ratio Meyn, Pandit, Veeravalli 200X

8 Goal: Robustness Original motivation for the research on mismatched detectors What is the impact on capacity given a poor model? How should codes be constructed to take into account uncertainty? X memoryless Y channel

9 Goal: Complexity Detection and capacity - easy. Its just mutual information!! Linear detectors suggest relaxation techniques for multiuser and network settings

10 Goal: Adaptation A new point of view! How can a detector be tuned on-line in a dynamic environment? X memoryless Y channel This will depend on SNR Other users Network conditions... Estimates Capacity

11 II Mismatched Channel X memoryless Y channel

12 Mismatched Detector A given function F (x,y) defines a surrogate ML detector: i =argmax i Empirical distributions for ith codeword: Γ i n = n 1 Γ i n, F =argmax i Reliably received rate (in general a lower bound): Generalized Mutual Information (Lapidoth, Csiszar) n t=1 δ X i t,y t n t=1 F (X i t, Y t )

13 Mismatched Detector: Sanov s Theorem Sanov s Theorem (in form of Chernoff s Bound) gives log P e ni LDP (P X ; F ) ( Pe pair-wise prob. of error) { } I LDP (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F

14 { Mismatched Detector: Sanov s Theorem Sanov s Theorem (in form of Chernoff s Bound) gives log P e ni LDP (P X ; F ) ( Pe pair-wise prob. of error) { } I LDP (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F π 0 Q β0 (π 0 )={ Γ : D(Γ ) β 0 π 0 = P X P Y π 0

15 { Mismatched Detector: Sanov s Theorem Sanov s Theorem (in form of Chernoff s Bound) gives log P e ni LDP (P X ; F ) { } I LDP (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F π 0 Q β0 (π 0 )={ Γ : D(Γ ) β 0 π 0 = P X P Y π 0 Γ, F = P XY, F P XY β 0 = I LDP (P X ; F )

16 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information - introduces typicality constraint, log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y }

17 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y } Derivation of GMI bound using Sanov:

18 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y } Derivation of GMI bound using Sanov: For any function G(y), i =argmax i { n } Γ i n, F + G =argmax [F (Xt, i Y t )+G(Y t )] i t=1 log P e ni LDP (P X ; F + G) any G(y)

19 Mismatched Detector: Generalized Mutual Information Generalized Mutual Information log P e ni GMI (P X ; F ) { I GMI (P X ; F ):=inf D(Γ P X P Y ): Γ, F = P XY, F, Γ 2 = P Y } log P e ni LDP (P X ; F + G) any G(y) sup I LDP (P X ; F + G) =I GMI (P X ; F ) G(y) G * : Lagrange multiplier for equality constraint Γ 2 = P Y

20 III Learning the Best Detector η ϱ β (η ϱ )

21 Learning the Best Test Given a parameterized class of detectors, What is the best value of α? {F α = α i ψ i : α R d } Maximize MM capacity: J(α):=I LDP (P X ; F α )

22 Learning the Best Test Maximize MM capacity: J(α):=I LDP (P X ; F α ) Derivatives: J = ψ Γ α, ψ 2 J = Γ α, ψψ T Γ α, ψ Γ α, ψ T where for any set A B(X Y), Γ α (A):= P X P Y, e F α I A P X P Y, e F α

23 Learning the Best Test Maximize MM capacity: Derivatives: J(α):=I LDP (P X ; F α ) J = ψ Γ α, ψ 2 J = Γ α, ψψ T Γ α, ψ Γ α, ψ T where for any set A B(X Y), Γ α (A):= P X P Y, e F α I A P X P Y, e F α Newton Raphson based on channel model, or Stochastic Newton Raphson based on measurements alone!

24 Learning the Best Test Estimation algorithm is decentralized and model-free X memoryless Y channel X Estimation α(t)

25 AWGN Channel Y = X + N SNR = Basis: ψ =(x 2, y 2, xy) T Estimates Capacity iterations

26 Mismatched Neyman Pearson Hypothesis Testing Numerical experiment: X 1 uniform on [0, 1], X 0 = 5 X 1 η β (η) False Alarm Error Exponent Missed Detection Error Exponent Stochastic Newton-Raphson to find [ η, β (η) ] pairs Parameterized class of detectors, {F α = α i ψ i : α R d } ψ(x) =[x,log(x)/10] T Optimal detector

27 Mismatched Neyman Pearson Hypothesis Testing X uniform on [0, 1], X 0 = 5 X 1 Stochastic Newton-Raphson η ϱ False Alarm Error Exponent β (η ϱ ) Missed Detection Error Exponent Parallel simulation for a range of η

28 Mismatched Neyman Pearson Hypothesis Testing X 1 uniform on [0, 1], X 0 = 5 X 1 β (η) ψ(x) =[x,log(x)/10] T ψ(x) =[x,x, 2..., x 6 ] T ψ(x) =[x, (1 + 10x) 4] ψ(x) =[x,x 2 ] T ψ(x) =[x,1/(1 + 10x)] T Optimal detector T 0.10 D(π 1 π 0 )= Parallel simulation for a range of η 2.4 η

29 IV Conclusions

30 Conclusions Geometry based on Sanov s Theorem combined with Stochastic Approximation provides powerful computational tools

31 Conclusions Geometry based on Sanov s Theorem combined with Stochastic Approximation provides powerful computational tools Future work: Optimizing the input distribution Application to MIMO will simultaneously resolve coding and resource allocation. Extensions to network coding possible? Simulation algorithm exhibits high variance. Application of importance sampling? Revisit robust approach?

32 References [1] E. Abbe, M. Médard, S. Meyn, and L. Zheng. Geometry of mismatched decoders. Submitted to IEEE International Symposium on Information Theory, [3] S. Borade and L. Zheng. I-projection and the geometry of error exponents. In Proceedings of the Forty-Fourth Annual Allerton Conference on Communication, Control, and Computing, Sept 27-29, 2006, UIUC, Illinois, USA, [4] I. Csiszár. The method of types. IEEE Trans. Inform. Theory, 44(6): , Information theory: [7] I. Csiszár and P. Narayan. Channel capacity for a given decoding metric. IEEE Trans. Inform. Theory, 41(1):35 43, [12] G. Kaplan, A. Lapidoth, S. Shamai, and N. Merhav. On information rates for mismatched decoders. IEEE Trans. Inform. Theory, 40(6): , [14] C. Pandit and S. P. Meyn. Worst-case large-deviations with application to queueing and information theory. Stoch. Proc. Applns., 116(5): , May [15] O. Zeitouni and M. Gutman. On universal hypotheses testing via large deviations. IEEE Trans. Inform. Theory, 37(2): , 1991.